python thrift两种服务模式对比

最新推荐文章于 2026-05-16 16:25:44 发布

原创最新推荐文章于 2026-05-16 16:25:44 发布 · 2.3k 阅读

1 ·

本内容遵循CC 4.0 BY-SA版权协议

标签

#python #thrift #多线程

杂谈专栏收录该内容

16 篇文章

订阅专栏

本文对比了两种Thrift RPC服务实现方案：一是直接在服务端进行计算，导致响应时间过长；二是通过多进程处理计算任务，显著提升了服务性能。

方案一

服务代码只是死循环一秒钟

import sys, glob  
sys.path.append('./gen-py')

from HelloService import HelloService  
from HelloService.ttypes import *  


from thrift.transport import TSocket  
from thrift.transport import TTransport  
from thrift.protocol import TBinaryProtocol  
from thrift.server import TServer  

class HelloServiceHandler:  
  def __init__(self):  
    self.log = {}  
  def func1(self):  
    print 'func1()'  
  def sayHello(self):  
    print 'sayHello'  
  def getData(self, input): 
      i = 0 
      while True :
          i += 1
          if i > 30000000 :
              break

      return input+' from server 1024';  


handler = HelloServiceHandler()  
processor = HelloService.Processor(handler)  
transport = TSocket.TServerSocket(port=9090)  
tfactory = TTransport.TBufferedTransportFactory()  
pfactory = TBinaryProtocol.TBinaryProtocolFactory()  

#server = TServer.TSimpleServer(processor, transport, tfactory, pfactory)  

# You could do one of these for a multithreaded server  
server = TServer.TThreadedServer(processor, transport, tfactory, pfactory)  
#server = TServer.TThreadPoolServer(processor, transport, tfactory, pfactory)  

print 'Starting the server...'  
server.serve()  
print 'done.'

测试程序

import sys, glob
sys.path.append('./gen-py')

from HelloService import HelloService
from HelloService.ttypes import *


from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol


import datetime

while True :
  sleep(1)
  try:
    starttime = datetime.datetime.now()
    # Make socket
    transport = TSocket.TSocket('localhost', 9090)

    # Buffering is critical. Raw sockets are very slow
    transport = TTransport.TBufferedTransport(transport)

    # Wrap in a protocol
    protocol = TBinaryProtocol.TBinaryProtocol(transport)

    # Create a client to use the protocol encoder
    client = HelloService.Client(protocol)

    # Connect!
    transport.open()

    client.sayHello();
    print 'ping()'

    print(client.getData("client access"))
    # Close!
    transport.close()

    endtime = datetime.datetime.now()
    print (endtime - starttime).seconds

  except Thrift.TException, tx:
    print '%s' % (tx.message)

测试结果

启动thrift服务，在四核心的电脑上，启动四个shell分别执行测试程序，结果：

client access from server 1024
12
ping()
client access from server 1024
16
ping()
client access from server 1024
17

每次调用的时间达到了16s左右（实际只是一个一秒左右的循环）

方案二（用multiprocessing）

from time import sleep
import multiprocessing

def work(pipe ,  b ) :
    i = 0
    while True :
        i += 1 ;
        if i > 30000000 :
            break
    pipe.send('saaa000')

class HelloServiceHandler:  
  def __init__(self):  
    self.log = {}  
  def func1(self):  
    print 'func1()'  
  def sayHello(self):  
    print 'sayHello'  
  def getData(self, input): 
      pipe = multiprocessing.Pipe()
      #print pipe
      # Pass an end of the pipe to process 1
      p1   = multiprocessing.Process(target=work, args=(pipe[0],input ))
      p1.start()
      p1.join()
      return pipe[1].recv()        

      #return input+' from server 1024';  


handler = HelloServiceHandler()  
processor = HelloService.Processor(handler)  
transport = TSocket.TServerSocket(port=9090)  
tfactory = TTransport.TBufferedTransportFactory()  
pfactory = TBinaryProtocol.TBinaryProtocolFactory()  

#server = TServer.TSimpleServer(processor, transport, tfactory, pfactory)  

# You could do one of these for a multithreaded server  
server = TServer.TThreadedServer(processor, transport, tfactory, pfactory)  
#server = TServer.TThreadPoolServer(processor, transport, tfactory, pfactory)  

print 'Starting the server...'  
server.serve()  
print 'done.'

测试结果

测试程序同上，结果

ping()
saaa000
2
ping()
saaa000
2
ping()
saaa000
2
ping()
saaa000
2
ping()
saaa000
2
ping()
saaa000
2
ping()

同样开四个终端同时测试，时间只需要2秒。从这个层面来看，用计算密集型的rpc调用用一个多线程的程序在前端做接收分发以及结果的返回，而将计算交给其它进程的方法是更优的，能够充分利用CPU的资源。