binary applications won't tell you anything if you just open them in a hex editor.. it'll just show you machine code.
You'd have to, at the least, decompile it. But that probably wouldn't help a lot as it'll be in assembly, the lowest level programming language.
If you want to learn more about programming binary applications and things like .dll's and such, you need to learn a slightly higher level language like C/C++.. Although they can be created in assembly too. I'd suggest getting a few books on C. Not any of the "Teach Yourself C in 21 days, or 24 hrs" crap.. those are mostly trash. For learning C I recommend "Pointers on C" by Kenneth Reek (ISBN 0-673-99986-6), for C++ I recommend "The C++ Programming Language" by Bjarne Stroustrup (ISBN 0-201-88954-4).. If you want to know more about the Win32 API: "Programming Windows" by Charles Petzold (ISBN 1-57231-995-X)
Also, there are tons of examples on the MSDN site, along with all the details of the Windows APIs, DirectX, and many more:
http://msdn.microsoft.com/library/