Skip to content

Commit ca6d7a8

Browse files
committed
Fix the problem that caused it to leave thread pools open after starting and flooding the system with dead processes
1 parent 3b7e262 commit ca6d7a8

File tree

1 file changed

+10
-2
lines changed

1 file changed

+10
-2
lines changed

content_processor.py

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -95,8 +95,16 @@ def processBody(self):
9595
l.append(self.text[i:j])
9696
i = offset + j+1
9797
#print "processing with %i threads" % len(l)
98-
pool = Pool(processes=(len(l)))
99-
self.keyword_dicts = pool.map(rankKeywords, l)
98+
try:
99+
pool = Pool(processes=(len(l)))
100+
self.keyword_dicts = pool.map(rankKeywords, l)
101+
except KeyboardInterrupt:
102+
pool.terminate()
103+
pool.join()
104+
sys.exit()
105+
else:
106+
pool.close()
107+
pool.join()
100108
#print "processed, returned %i dicts" % len(self.keyword_dicts)
101109
else:
102110
self.keyword_dicts.append(rankKeywords(self.text))

0 commit comments

Comments
 (0)