 |
||
|
C#/.NET vs Windows Native Code | Windows API vs. C Run-Time Library | Comments on the Windows API | Pthreads Emulation Edition 4 and Edition 3 Errata and CommentsDownload Examples Code. - Version 1.16 April, 2015Last Update - October 28, 2015 (newer entries in red)I'd like to express my appreciation to the many readers who have contributed to the information here, and I've cited them whereever possible. Thank you for your assistance and your patience; the contributions have been extremely helpful - JMH. This page contains: A Little Humor and Poetry Before Getting SeriousOctober 28, 2015 An Ode to Code: The Windows Thread that Wouldn't Go to Bed. If you've read Chapters 7 and 8, you might appreciate this. If you have not gotten that far, you can see what awaits you. My hat's off to Anirudh Sahni for this effort. Errata, Comments on Windows System Programming, Fourth EditionFebruary 26, 2013. Windows and Kernel Versions. Windows 8 and Windows Server 2012. Pages 3-5 (and elsewhere) discuss Windows version as well as the kernel versions. Since publication, Microsoft has released the new versions listed above, which all use the updated Kernel, NT 6.2. Visual Studio 2012 is also available. The impact on the book's topics is minor but worth mentioning.
In all cases, defects discussed in this section are fixed in the s, defects discussed in this section are fixed in the Examples file. See the ReadMe.txt for a brief explanation of the latest updates. May 17, 2010. Appendix C program execution times. See the Appendix C entry below for an important comment about these results. May 17, 2010. Building examples as C++. Gavin Brewer pointed out that there can be build problems if you build some example files as C++ rather than C. This can happen, for example, if you change the source file suffix to .cpp from .c. For more information, go to the Chapter 10 entry. March 30, 2010. Kwan Ting Chan (KTC), along with several others, has pointed out that I do not use generic strings and characters consistently in the Examples programs or in the book. There are many examples of this inconsistency; I have fixed a large number in Examples Version 1.04 (I don't guarantee I found them all). KTC has done a very thorough job, and, to use one of her favorite quotes, "Experience is a good school but the fees are high." (Heinrich Heine). Also, see the ReadMe.txt file in the Examples file, including the new Bullet 5. March 26, 2010. Program Bugs and Enhancements. Explanations with the chapters. March 5, 2010. Windows 7. I've tested nearly all example programs on Windows 7; at the time of publication, only the Release Candidate was available. As one would expect, there have been no problems. Furthermore, the performance results are consistent with those in Appendix C. Nov 17, 2010. This remains true after much more experience. March 5, 2010. Visual Studio 2010. I've converted and tested several examples without any problems. Nov 17, 2010 and later. This remains true after much more experience. March 10, 2010. Chapter 1, all programs, Bug. Balbir Singh <bsingharora@gmail.com> pointed out (quite correctly) that the introductory file copy programs set a bad example for two reasons:
The Version 4 Examples file fixes the problems. Interestingly, I've used these examples for years, sometimes as a warm up exercise in classes with very experienced engineers, and no one has ever complained. That does not make it right! March 10, 2010. Chapter 2, Program 2-1. Mid page. The _T macro was omitted in the _tfprintf statement after the eMsgLen test. March 26, 2010. Chapter 2. Program 2-4 (and cci variations). BUF_SIZE is 256, which is too small for efficient operation. 16384 or larger (use powers of 2) is about optimal. I've changed all the Examples cci variations as well as the cpW program. This also applies to Program 1-2 (cpW). April 24, 2011. Chapter 2, Program 2-5 (page 54, mid-page). The two statements after the if statement should be indented and enclosed in braces. { va_end(pMsgList); return FALSE; } The code in the Examples file is correct. Dylan Tu found this error. March 10, 2010. Chapter 3, Page 63, Last Line. Comment. GetDiskFreeSpace() is now used in Chapter 5's version program (Page 180) to display disk sector and cluster size. MSDN says that you should use GetDiskFreeSpaceEx() for disks larger than 2GB (which would be nearly any modern device). However, the extended function returns different information, and GetDiskFreeSpace() appears to work well on large disks. March 10, 2010. Chapter 3, Page 65, Line 5. Mistake. SetEndOfFileEx does not exist; there is no need for an extended function. Use SetEndOfFile. March 26, 2010. Chapter 3, pp 75ff, Program 3-2 (lsW). Michael Bruestle (Austria) made a really important point. The TraverseDirectory function maintains the current directory in a local variable which is kept on the stack during the recursive traversal. This is very poor practice as it consumes a lot of space and there is a risk of stack overflow (which, in turn, is a security risk - very bad). The problem would be aggravated and severe if we allowed \\?\ prefixed long path names (the current limit is MAX_PATH or 256). There's a corrected version in the Examples file (Version 1.0.3, posted today) which corrects this problem and is also a lot simpler than the original. The new version has a single array for the path name, which grows during the traversal and is restored after each recursive call. Incidentally, lsReg (Program 3-4) and lsFP (used with Program 15-2) have the same problem; I did not fix them (an exercise for the reader using the same techniques as in lsW). July 8, 2010. Chapter 3, pp 75ff, Program 3-2 (lsW). Petr Kuzmic <http://www.biokin.com> pointed out that recursive listings through deep directory structures will someitmes omit the backslash and sometimes have two backslashes in a row between directory names. Petr's fix was straightforward, and I added a few more adjustments. Also, please note that this program is intended to illustrate the Windows directory processing API and is not intended to be a faithful implementation of the POSIX ls command. Nonetheless, this fix and the March 26 fix are important. April 5, 2013. Chapter 3, pp 75ff, Program 3-2 (lsW). About Line 134. Use SetCurrentDirectory(_T("..")); as suggested in the comment (but is correct in the book) for correct operation with relative paths. I also added a local variable (subdirectoryPath) rather than overwriting a function parameter. I'm grateful to Liyan, a Chinese reader, who identified this bug. January 4, 2017. Chapter 3, pp 79ff, Program 3-3 (touch). Gao Jingsheng pointed out that the variable maFlag is not needed and is used incorrectly so that the access and modifiy times are set if both the -a and -m flags are off. The problem came from a careless change made years ago. This is fixed in Version 1.18 of the examples file. March 10, 2010. Chapter 5, Page 180, Exercise Run 5-12, GetDiskFreeSpace() is now used to add a display of disk parameters on the current disk. See the Page 63 comment above. January 22, 2012.>January 22, 2012. Chapter 5 (pp 150ff) and elsewhere. CreateFileMapping() returns NULL to indicate an error, not INVALID_HANDLE_VALUE. Unfortunately, some code examples, such as cci_fmm.c and cci_fmmDll.c, tested for INVALID_HANDLE_VALUE. Version 1.14 of the examples corrects these errors. March 26, 2010. Chapter 6, Page 203, Program 6.2 (timep). Michael Bruestle (Austria) made the point that there should be a decimal point rather than a colon between the seconds and milliseconds on the output. Done. Also note the useless call to GetSystemTime (you don't need startTimeSys either). This is an artifact of the old Windows 9x code where GetProcessTimes is not available, and I computed elapsed time by subtracting start tiem from end time; I'm glad the Windows 9x is in the rear view mirror! March 17, 2012. Chapter 6, Page 211. Job Management. The examples file, JobMgt.c, DisplayJobs() function, Line 110, has an inexplicable error that I can't believe I made (but, there it is). Reev A. Edd noticed this and brought it to my attention. The Boolean function GetJobMgtFileName() is treated as an integer. Change the line to: if ( !GetJobMgtFileName (jobMgtFileName) ) Also, make a similar change at Line 181 in function FindProcessId(). October 18, 2011, Chapter 6, p. 208, Program 6.3. The two calls to GenerateConsoleCtrlEvent near the bottom of the page (in the Kill function) are wrong, and the calls will report an error (ERROR_INVALID_PARAMETER). Replace the ProcessId argument with 0; see the MSDN entry for the explanation. Thanks to Nathan Myers (ncm@cantrip.org) for reporting this mistake. This fix applies to both JobShell and JobObjectShell. I will post the corrected code shortly; I need to perform some additional testing first. November 17, 2010, Chapter 9, p. 330. Note that the correct signature for GetProcessAffinity mask requires that the types for the second and third parameters be PDWORD_PTR. May 17, 2010. Chapter 10, pp 348ff, Programs 10-3, 10-4, 10-5, building as C++. These comments are not limited to these examples. Gavin Brewer created C++ synchronization classes as suggested in Exercise 10-7. When he compiled the resulting code, he got errors such as the following for every _beginthreadex call: error C2664: '_beginthreadex' : cannot convert parameter 3 from 'DWORD (__stdcall *)(PVOID)' to 'unsigned int (__stdcall *)(void *)' Parameter 3 is the thread function; in this instance, the thread function is decclared as: DWORD WINAPI Producer (PVOID); _beginthreadex requires parameter 3, the thread function, to return an unsigned. If you dig around in the Windows include files, you'll find that DWORD is actually defined as an unsigned long. Specifically, I looked in:
C:\Program Files\Microsoft SDKs\Windows\Later, I'll update the source files to avoid this problem.
Also notice that you need the header file Synchapi.h for Windows 8 and Windows Server 2012. See MSDN for more information. May 17, 2010 Appendix C, Tables C-1, C-2.ss="style38">May 17, 2010 Appendix C, Tables C-1, C-2. Conor O'Rourke pointed out that the statement at the bottom of p. 577 about clearing physical memory before each test run is not correct; specifically, memory is not cleared. Consequently, the files are, in general, cached in memory and can be considerd to be "live". A quick look at the results confirms that the times are generally far too small to allow for reading and writing of files or typical disks and I/O systems. For example, my system #6 uses a SATA 2.0 system with a maximum throughput of about 300 MB / second. November 17, 2010, Appendix C, Table C-2 and others, Column 1 (XP laptop). This table is misleading and unfair to XP; some of the performance figures are awful. For example, see the cciMT and cciOV entries, as well as wcMT in Table C-3. The reason for the bad performance is that the system has only 1.5 GB RAM, which is not sufficient for all the data to be concurrently memory resident, resulting in high page fault rates. Tests on XP systems (including 64-bit multi-core systems) with more memory provide much better results. Nonetheless, the printed results are valuable as they show the importance of having sufficient memory for some processing tasks. I've generally found the performance numbers to be more valuable when using live files. First, it's common in real applications that you are already using a file before you start other processing, and I've found these comparisons to be useful. Second, you do get a meaningful comparison between different implementation strategies.
Download PowerPoint Overheads and Exercises.These are offered "as is" for non-commercial, instructional use. I'm also available for on-site professional training using this material. For information, please contact me at jmhart62@gmail.com. Windows System Programming, Edition 3Windows System Programming, Edition 4 (Published February, 2010) addresses all appropriate issues in this section. Also, see the Edition 4 section above; some of those comments may apply here. DOWNLOAD Version 1.10 (January, 2008) (about 1.7 MB) from here. Principal change (January 2008): Chapter 12's SendReceiveSKST.c, with the projects rebuilt. Also, Chapter 10 has an improved version of the ThreeStage.c program written for Pthreads. See the ReadMe.txt file for additional changes. Additional Note for Visual Studio 2005 and 2008 users: VS2005/8 will convert the VS .NET solutions when you open the solution. Aug 11, 2009. New projects will be posted later this year -- They are posted in the Edition 4 Examples. Notes for NT6 (Vista, Server 2008, Windows 7) Users
Notes for 64-bit Programmers (XP, Vista, Server 2003, 2008, Windows 7)
Errata, Comments on Windows System Programming, Third EditionJune 5, 2005. Projects7 (Visual C++ 7.0 projects directory). Several projects were set up to build to the run6 directory rather than run7. This is fixed in the 1.4 code download. Affected projects are: asc2unDLL, JobShellSecure, sortMM (debug build only), and Utility_3_0 (debug build only). Thanks to Bill Stanton for finding this error. Chapter 1, p. 16, bullet 4. March 31, 2007. Aaron Yang points out an important misstatement. fread and fwrite return the number of objects, not the number of bytes, read or written. This does not affect Program 1-1 because the object size is one. Chapter 2. May 16, 2005. I mention here and in Appendix C that FILE_FLAG_SEQUENTIAL did not seem to help performance significantly. However, I recently chatted with a colleague who has processed large files (a gigabyte or so) and has found a significant benefit from this flag. I'll try to pin down more information. Chapter 2, p. 28. April 17, 2006. YangYang, from ChengDu, China, points out that creating a file with TRUNCATE_EXISTING fails if the file does not exist, contrary to the statement in the book. Experiments quickly confirm this fact, which is also stated in MSDN. Furthermore, this is the expected behavior, considering the option's name. Chapter 2, p. 32. April 17, 2006. Bullet 4, Line 3 (5 lines from the bottom of the page). It says "_tstcpy (for strcpy)". This is incorrect. and should say "_tcscpy (for strcpy)". Chapter 2, p. 42, June 7, 2005. Program 2-2. I've used ReportError() throughout. However, you may prefer to use the standard strerror() function to generate text error strings. Chapter 2, p. 48. May 16, May 28, 2005. The reference to BY_HANDLE_FILE_INFO is incorrect; it should be BY_HANDLE_FILE_INFORMATION. Also, MOVEFILE_WRITETHROUGH, used with MoveFileEx, should be MOVEFILE_WRITE_THROUGH. Thanks to Paul Narula (narula@math.utexas.edu) for identifying this mistake. Paul pointed out an omission in the MoveFileEx discussion on p. 48., for which I apologize and thank Paul. In his words:
Chapter 3, pp 59-60. Dec 19, 2004. The references to the function SetValidFileData are incorrect. There is no such function; the correct name is SetFileValidData. Thanks to Simon Bailey for finding this error. Chapter 3, p. 62 (Program 3-1), May 28, 2005. Thanks to Bill Stanton for finding this important defect. In the two QuadPart calculations, it's important to cast at least one of the multiplication operators to LONGLONG to assure 64-bit arithmetic. Casting sizeof(RECORD) is sufficient. Thus, we have lines 82 and 106 of RecordAccess.c changed to:
Incidentally, I completed a project recently that required huge file processing, and everything worked well using code similar to this example (of course, I did the 64-bit arithmetic properly). However, the error in the book did not occur to me until Bill pointed it out. Chapter 3. p. 69 (Program 3-2, lsW.c), Oct 9, 2005. TempPath is not initialized; it should be initialized the same way that PathName is. This problem is corrected in Version 1.7 of the download zip file. Shin-Wei Hwang reported this problem and the fix. p. 72 (Program 3-3), Sept. 5, 2005. Thomas Beard points out that the logic for SetAccessTime and SetModTime is reversed. Furthermore, there are some other logic changes to emulate the UNIX version's error reporting (or lack thereof) correctly. The corrected code can be downloaded as Version 1.6. Chapter 4, pp. 107-109 (Program 4.2). Robert Hoggard pointed out an inconsistency in this program. The file is open with no share mode, but then it is locked. Why bother to lock it if it can't be shared? The first answer is that the locking is redundant and can be removed, and the lock should always succeed, so there is no need to test it. However, it would be possible for another process to inherit the open file handle if you are creating new processes (Chapter 6), or another thread (Chapter 7) might access the file through the open handle (a very unsafe thing to do!). In this simple example, however, you could remove the lock and unlock. Chapter 4, pp. 114-115 (Program 4-5). June 23, 2007. A reader points out that the second print statement ("Leaving handler in 6 seconds" ) rarely appears. This is because the main program, running in a separate thread from the handler, samples the Exit flag every 5 seconds and usually exits before the handler can print the separate message. It would be more accurate for the first message to say ""Ctrl-c received by handler. Leaving in 5 seconds or less." and for the second to say that it will be one second or less. Furthermore, it would be best to set Exit in the individual case statements, as you might not want to terminate the program in all situations. The source file (CHAPTR04\CNTRLC.C) is fixed in the downloadable solutions. Chapter 5, p. 136. Sept 6, 2006. Eric Berge says, correctly, that there is an error in the description of OpenFileMapping() on page 136. The dwDesiredAccess values are not the same as dwProtect in CreateFileMapping() but rather should be the same as the dwAccess values to MapViewOfFile()." This makes sense as you are specifying the access that you wish to have, and dwDesiredAccess is checked against the security descriptor of the named file mapping object. Chapter 5. Nov. 21, 2004. randfile.c has been updated so that all generated lines terminate with CR, LF. The files can then be used with programs such as sortBT.c, which require data files with CR, LF terminated lines. August 7, 2005. Matthew Wilson, the author of Imperfect C++, makes an interesting point about memory mapping handles, and I'd neglected this point in Chapter 5. "Just a quick note about the use of memory mapped files, specifically wrt the timing of CloseHandle(). A file mapping object maintains a reference to the file which it maps, so one can call CloseHandle() on the file object as soon as CreateFileMapping() has succeeded (or failed, for that matter). Similarly, a view maintains a reference to the file mapping, so one can call CloseHandle() on the file mapping object as soon as MapViewOfFile(Ex)() is called. In this way, one need maintain only a single 'handle' (the void* pointer to the view) which is especially useful for creating C++ wrapper classes utilizing simple RAII (as seen in the WinSTL memory_mapped_file class, part of the STLSoft libraries)." JH Note: "STL" is the C++ Standard Template Library. Chapter 7 (All), November 16, 2008. I've done some work with Cilk++ from Cilk Arts, Inc (now owned by Intel). ("The Cilk++ cross-platform solution offers the easiest, quickest, and most reliable way to maximize application performance on multicore processors. Cilk++ provides a simple set of extensions for C++, coupled with a powerful runtime system for multicore-enabled applications. Cilk++ enables rapid development, testing, and deployment of multicore applications."). Cilk++ makes it very easy to write programs to exploit application parallelism, and I've written a blog on the Cilk Arts site to show how to write the Cilk++ verion of the wc example (used to solve exercise 7.6). Performance scales almost linearly with the number of processors, and the Cilk++ code greatly simplifies the thread management. Also, take a look at the conding of the word count function; it's much faster than the version in the book solution file. Chapter 7, pp 207-208, and Appendix A, pp 480-481. June 20, 2007. Clive Darke asked about the bottom of page 207, which tells you to #define _MT, as does the Appendix A listing of Envirmnt.h. However, if you use Visual Studio to set the "Mutlithreaded DLL" C++ code generation options, _MT will be defined for you. This is the correct way to define this macro. September 14, 2008. Chapter 8, p 256, about 2/3 down, a comment says "As in Program 9.1". This should be Program 8.1. Chapter 9, p. 266-268, Sept 5, 2005. Jemzer Ulrich noted that the first three lines on the top of p. 268 are in the wrong order. The return 0; should obviously come last. This applies to statsMX.c, statsCS.c, statsIN.c, and statsNS.c. There is no practical impact, however, as process termination will free all the resources. Chapter 9, p. 273. June 7, 2005. Bill Stanton pointed out a significant error in the code fragment at the top of the page. The semaphore wait and release should be outside the loop, as in the TimedMutualExclusion example program and as follows:
The original form would simply aggravate the situation with additional waits. In this new form, the number of active worker threads contending for the mutex is limited. An alternative, which would still limit the number of active threads but which would be fairer by preventing a single worker from hogging the semaphore, would be the following (I have not tried this in TimedMutualExclusion, but doing so would be an interesting experiment - I'd be interested in hearing from anyone who does try it):
Chapter 9, p. 275. June 7, 2005. Second section ("System, Process, and Thread Affinity Masks", second bullet, should read "The process mask ...". Thanks, again, to Bill Stanton. December 30, 2005. Program 10-2, Page 290. Shin-Wei Hwang noticed that the CreateThresholdBarrier() function uses an auto-reset event; it should use a manual-reset event, which is appropriate for the "broadcast model". Line -6 should be: hthb->b_broadcast = CreateEvent (NULL, TRUE /* Manual-Reset. */, The program on the disk/web site used the signal model, which also works (each release thread signals to release another thread). However, I've change the code to be consistent with the code in the book. December 30, 2005. Page 293, last sentence of the first full paragraph. Replace the last sentence with, "The first two functions provide placeholder values and logic for time outs, which could be a useful additional feature." Chapter 10. August 14, 2005. Program 10.4, Page 295. Serious error in q_full(). The logic is clearly wrong and has been fixed in the download code. The problem was spotted by some sharp-eyed course participants at Ask Jeeves (Now, just Ask; as a "Jeeves and Wooster" fan, I always liked the original name). The correct code for the function is: DWORD q_full (queue_t *q) It's remarkable that this bug. which reversed the use of the first and last indices, was undetected for such a long period of time - very embarrassing! Thanks to Kevin Russo for pointing out that the new code was not posted properly. It is now fixed in the download code. September 14, 2008. Program 10.4, p. 296. Kevin Russo points out two things. First, q_remove() should test for an empty queue and return an error immediately if the queue is empty. This preserves the symmetry with q_insert(); this is another one of those long-lived bugs (perhaps a cockroach) that has survived through years of use and testing, perhaps because the fault scenario never occurred. While cleaning thing up, Kevin suggested that I put the empty (full) test at the beginning of q_remove() (q_insert()) to make things a little faster. I've implemented this; it's a good idea and good practice. Chapter 10, January 28, 2008. David Harding asked about the Pthreads version of ThreeStage.c; I have added an updated version of the code to the download file. You can run it using the open source Pthreads library. Chapter 10. Good News, May 13, 2009 and September 14, 2008. Windows now has a real condition variable, much like Pthreads. Of course, the API is different from the Pthreads API. However, you need the Windows NT6 kernel, so you cannot use them on Windows XP and Windows 2000/2003, so their utility is very limited at this time (you would be limited to Vista or Windows 7 on the client side). A server would require Windows Server 2008. I tried the Windows Condition Variables on the ThreeStage QueueObj functions, and everything works fine and does provide good performance. You use the CS with CRITICAL_SECTION objects. You can also use them with "slim reader/writer locks", another new feature requiring NT6 (Vista, Server 2008, W7). The MSDN example uses queues with "not full" and "not empty" CVs, so the MSDN examples will be familiar. What is more, MSDN explicitly refers to the pulse/manual-reset and set/auto-reset analogy. Chapter 11, Figure 11-1. August 14, 2007. Program 1 on the bottom left of the figure contains a typo. The second line should be: hWrite = GetStdHandle (STD_OUTPUT_HANDLE); This is symmetrical to Program 2 (P2), so that the pipe is known as hWrite in P1 and hRead on P2. Likewise, P1 takes input from hIn, and P2 sends output to hOut. Thanks to Greg Gagne, coauthor of the Operating Systems Concepts series (Silberschatz, Galvin, and Gagne) who brought this to my attention. Chapter 11. Program 11-3 (serverNP). March 5, 2005. First, there's a minor difference between the code in the book and on the disk. About 2/3 down on p. 332, interchange the two lines, CloseHandle(hConTh); and if (ShutDown) continue; Also, we need to test hConTh == NULL before caling GetExitCodeThread(). Futhermore, hConTh is not closed, although the system will shut down, so it should not be a big problem. More seriously, however, is a problem that Scott Drummonds (www.e-scott.net) pointed out. My technique to shut down stray connection threads is, well, a bit of a kludge. As Scott points explains,
Now, the code on the disk has several suggested alternatives, but none of them seem quite right. Here's my suggestion, not yet tested (watch this space!), which uses asynchronous I/O, which doesn't get covered until Chapter 14. Here's an idea which allows the connection thread to poll the Shutdown flag. Create the named pipes with FILE_FLAG_OVERLAPPED. Then, modify Connect() to use an overlapped structure (of course, we haven't covered overlapped I/O at this point in the book). Then, in Connect(), you can wait on the manual reset event with a time-out, which allows you to loop and poll Shutdown. Thoughts or comments are appreciated; I've been busy and have not been able to try this out. January 28, 2008 and May 28, 2006. Program 12.4, Page 368-370. First, Armando Fortuna and Wangzhi have pointed out that the DLL_PROCESS_DETACH case should free the TLS index: TlsFree(TlsIx); before the return statement. NOTE: I would expect that the TLS index would be freed when the process terminates. However, I haven't been able to find any documentation that specifically says that this happens. Furthermore, it's good, safe, programming practice to free all resources specifically. May 29, 2007. Wangzhi reported that the zip file on the web site was not been updated; there is now a new version with this code change. It's version 1.9. I apologize for the omission. Next, Jun-Gil Jeong pointed out a problem in the ReceiveCSMessage function, which is now fixed in the download file. The nRemainRecv -= nXfer; statement is not needed before the comment: /* Transfer to target message up to null, if any */ Chapter 13, Nov. 21, 2004. Severity A. serviceSK.c. See the comments in the source code (version 1.2 download). The checkpoint is now updated reliably and the service is also properly set the the started state. There are several other improvements as well. Thanks to Steve Gibson for identifying the problem and suggesting the Edition 3, Chapter 14. See the important note below about atouMT. April 13, 2005 - Problem FIXED. Download code to get the fix; see the notes below for an explanation. BACK TO TOP |
||
 |
||
