Changeset 32321 for main/trunk
- Timestamp:
- 2018-08-02T21:50:16+12:00 (6 years ago)
- File:
-
- 1 edited
Legend:
- Unmodified
- Added
- Removed
-
main/trunk/gli/src/org/greenstone/gatherer/util/Readme_Using_SafeProcess.txt
r31720 r32321 924 924 925 925 926 926 _____________________________________________________________________________________________________________________ 927 928 P. STREAM GOBBLER **THREADS** ARE ALLOWED TO BLOCK (JVM WILL TIMESLICE). CAUSE INTERRUPTEDEXCEPTIONS TO END BLOCKS 929 _____________________________________________________________________________________________________________________ 930 931 In the InputStreamGobbler Thread classes for handling the proc.errstream and proc.outstream, the run() methods have a loop on readLine(): 932 933 while (!this.interrupted() && (line = br.readLine()) != null) { ... } 934 935 This so far worked beautifully, since if a process was cancelled, SafeProcess would call gobbler.interrupt() and the above loop would terminate. If a process had ended and the process err and output streams therefore run out, line == null and the loop would terminate. In both cases, we're out of the while loop and the process' stream would have been proeprly handed and the clean up would have taken place. 936 937 However, BufferedReader.read() and consequently its readLine() are actually blocking calls. This was discovered when the OpenOfficeExtension was included in a GS installation and didn't work properly: if it fails to launch soffice --headless to accept on some port, then soffice is not running headless in the background. At this point, the stderr stream of the perl process that launched the OpenOfficePlugin (and OpenOfficeConverter) worked fine and output the pluginfo for the plugin and finished, terminating the while loop for the err stream gobbler. However, the process' output stream gobbler never got any data, including eof(). No cancel button was pressed, so this.interrupted() wasn't true. And br.readLine() wouldn't return but blocked: it waited for data that never came. This blocked SafeProcess' call to join() on that gobbler thread, as its run() method never terminated. 938 939 The first solution is checking bufferedReader.ready() BEFORE attempting to any read() operation on the BufferedReader: 940 941 // readLine() can block if nothing is forthcoming, including eof marker. 942 // So must check BufferedReader.ready() before attempting readLine() on it 943 // See https://stackoverflow.com/questions/15521352/bufferedreader-readline-blocks 944 while ( !this.isInterrupted() && br.ready() && (line = br.readLine()) != null ) { ... 945 946 947 HOWEVER, there are 2 problems with this: 948 949 1. As per the API, ready() returns "True if the next read() is guaranteed not to block for input, false otherwise. Note that returning false does not guarantee that the next read will block." 950 951 NOTE: As per Andrew Mackintosh the above does not mean what I understood it to mean. I thought it means that if a read() blocks at this moment and therefore ready() returns false now, it need not meant it will block 5 mins from now if there was finally data sent to the process' stdout 5 mins from now. Andrew says that what the above means is that if ready() returns false a read() operation may OR may not block at present, whereas if ready() returns true, doing read() is guaranteed to NOT block. 952 953 2. In the example of the OpenOfficeExtension, what happened is that the process' exit value was always 141, non-zero hence not a success. 141 signifies a SIGPIPE error: 954 955 // NOTE: exitValue can be 141 when we got nothing not even eof from any of the process' 956 // output streams. According to http://tldp.org/LDP/abs/html/exitcodes.html 957 // an exit value of 128+n means Fatal error signal "n". 128 + 13 = 141, so n = 13. 958 // And "Signal 13 is SIGPIPE" see https://github.com/Tunnelblick/Tunnelblick/issues/272 959 // 'SIGPIPE is the "broken pipe" signal, which is sent to a process when it attempts 960 // to write to a pipe whose read end has closed (or when it attempts to write to a 961 // socket that is no longer open for reading), but not vice versa. The default action 962 // is to terminate the process.' as per https://www.quora.com/What-are-SIGPIPEs 963 964 965 Having realised that br.read() itself can block and how to get such blocking gobbler threads to terminate, I discussed the problem with Andrew. He explained the solution: 966 967 - blocking IO calls in loops is FINE as long as they are in a separate thread and not the main thread: the OS, or the JVM in this case rather, will know to use timeslicing and set the processor on other tasks until blocking IO calls are ready again. 968 969 - to "wake" up blocking calls on process termination, the "Scheduler" parent class that launched the child process and the StreamGobbler Threads should cause InterruptedExceptions on its StreamGobbler threads once the child process has exited. The Scheduler should internally induce InterruptedExceptions on the StreamGobblers regardless of whether the process terminated naturally or was cancelled by the user. The Exception would then end the blocking br.readLine(). 970 971 In my case, the Scheduler is SafeProcess.java and is the interrupting Thread that is to send interrupted exceptions. So after doing 972 exitVal = proc.waitFor() 973 the scheduler class SafeProcess should somehow cause InterruptedExceptions on its gobbler threads. 974 975 I found these pages, which seem to imply indicate that the interrupting thread (SafeProcess in my case) should close the streams of the child process and that this will generate the necessary InterruptedExceptions stopping the read()/readLine() operations: 976 977 - https://stackoverflow.com/questions/3595926/how-to-interrupt-bufferedreaders-readline (example speaks of bufferedreaders reading from sockets rather than from a process' IO streams) 978 - https://arstechnica.com/civis/viewtopic.php?t=259678 979 980 I can't get it to work yet though, as at 1 and 2 Aug 2018. 981 982 983 ANOTHER IMPORTANT LESSON FROM ANDREW: 984 Andrew Mackintosh further said that the read() block calls in worker Threads are far better in terms of not wasting processor power than a constantly active while loop in the worker Thread. 985 1. So doing this: 986 987 while(!thisthread.isInterrupted() && br.read() ) // br.read() possibly blocks 988 989 is good since if the read() the blocks, the OS or JVM knows to reuse the processor for other tasks. (I think of it as the OS/JVM considers blocking calls as being idle processes and therefore puts other tasks onto processor in the meantime.) 990 991 Whereas: 992 while(!thisthread.isInterrupted()) { 993 if(!br.ready) continue; // will keep testing interrupted and ready() and so keeps using the processor. 994 else br.read(); 995 } 996 will keep the processor/core wastefully occupied even though there may be long stretches when there's no data to read. 997 998 999 UNFORTUNATELY, 1000 https://stackoverflow.com/questions/3595926/how-to-interrupt-bufferedreaders-readline 1001 says that BufferedReader.readLine() can't even be interrupted. And their solution doesn't work either. 1002 1003 Other links: 1004 - EOF/EOS (End of Stream) is indicated by a return value of -1 for read and null for readLine(). 1005 https://stackoverflow.com/questions/36569875/how-does-bufferedreader-readline-handle-eof-or-slow-input 1006 https://stackoverflow.com/questions/3714090/how-to-see-if-a-reader-is-at-eof
Note:
See TracChangeset
for help on using the changeset viewer.