iDVD burns standard media DVDs. DVD Player plays standard DVDs and .iso disk image files created from standard media DVDs. For all practical purposes, your homemade DVDs are standard media DVDs. You have every reason to expect that the Windows DVD player will handle them without incident.
The only caveat is that DVD-R content is recorded by a phase change in a light sensitive film. Commercial DVDs record content in micropits stamped a metallic film. For this reason, homemade DVDs tend to be sensitive to the particular model DVD drive used to read/play them. This is not a Mac/Windows issue. This is an issue of the specific DVD drive hardware and the specific line of DVD-Rs.
You cannot pick a random spindle of DVD-Rs off the shelf of the random drug store and expect them to function properly in the random DVD-R drive. You must test the specific combination of DVD burner, DVD-R media, and DVD player to ensure that it works.