网络编程基础
<table style="height: 30px; background-color: #afeeee; width: 1266px; ; width: 1266px;" border="0"> |
描述 |
---|
描述 |
---|
描述 |
---|
描述 |
---|
server端:
2.socketserver?
为了简化socket的开发,socketserver提供了多种socket服务器模块,关系如下:
| |
SocketServer内部使用IO多路复用、以及 “多线程” 和 “多进程”(后续篇章会提到),从而实现并发处理多个客户端请求的Socket服务端。即:每个客户端请求连接到服务器时,Socket服务端都会在服务器是创建一个“线程”或者“进程” 专门负责处理当前客户端的所有请求。
1.ThreadingTCPServer介绍:
ThreadingTCPServer是socketserver中最常用的模块,它是多线程,并且支持多并发。
使用ThreadingTCPServer:
- 创建一个继承自 SocketServer.BaseRequestHandler 的类
- 类中必须定义一个名称为 handle 的方法,用于处理客户端的交互。
- 启动ThreadingTCPServer
一个简单的TreadingTCPServer例子如下:
TreadingTCPServer源码解析:
模块关系:
内部调用流程为:
- 启动服务端程序
- 执行 TCPServer.__init__ 方法,创建服务端Socket对象并绑定 IP 和 端口
- 执行 BaseServer.__init__ 方法,将自定义的继承自SocketServer.BaseRequestHandler 的类 MyRequestHandle赋值给self.RequestHandlerClass
- 执行 BaseServer.server_forever 方法,While 循环一直监听是否有客户端请求到达 ...
- 当客户端连接到达服务器
- 执行 ThreadingMixIn.process_request 方法,创建一个 “线程” 用来处理请求
- 执行?ThreadingMixIn.process_request_thread 方法
- 执行 BaseServer.finish_request 方法,执行?self.RequestHandlerClass() ?即:执行 自定义 MyRequestHandler 的构造方法(自动调用基类BaseRequestHandler的构造方法,在该构造方法中又会调用 MyRequestHandler的handle方法)
相关模块源码
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Base class for server classes.
Methods for the caller:
- __init__(server_address,RequestHandlerClass)
- serve_forever(poll_interval=0.5)
- shutdown()
- handle_request() # if you do not use serve_forever()
- fileno() -> int # for select()
Methods that may be overridden:
- server_bind()
- server_activate()
- get_request() -> request,client_address
- handle_timeout()
- verify_request(request,client_address)
- server_close()
- process_request(request,client_address)
- shutdown_request(request)
- close_request(request)
- handle_error()
Methods for derived classes:
- finish_request(request,client_address)
Class variables that may be overridden by derived classes or
instances:
- timeout
- address_family
- socket_type
- allow_reuse_address
Instance variables:
- RequestHandlerClass
- socket
</span><span style="color: #800000;">"""</span><span style="color: #000000;">
timeout </span>=<span style="color: #000000;"> None
</span><span style="color: #0000ff;">def</span> <span style="color: #800080;">__init__</span><span style="color: #000000;">(self,server_address,RequestHandlerClass):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Constructor. May be extended,do not override.</span><span style="color: #800000;">"""</span><span style="color: #000000;">
self.server_address </span>=<span style="color: #000000;"> server_address
self.RequestHandlerClass </span>=<span style="color: #000000;"> RequestHandlerClass
self.</span><span style="color: #800080;">__is_shut_down</span> =<span style="color: #000000;"> threading.Event()
self.</span><span style="color: #800080;">__shutdown_request</span> =<span style="color: #000000;"> False
</span><span style="color: #0000ff;">def</span><span style="color: #000000;"> server_activate(self):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Called by constructor to activate the server.
May be overridden.
</span><span style="color: #800000;">"""</span>
<span style="color: #0000ff;">pass</span>
<span style="color: #0000ff;">def</span> serve_forever(self,poll_interval=0.5<span style="color: #000000;">):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Handle one request at a time until shutdown.
Polls for shutdown every poll_interval seconds. Ignores
self.timeout. If you need to do periodic tasks,do them in
another thread.
</span><span style="color: #800000;">"""</span><span style="color: #000000;">
self.</span><span style="color: #800080;">__is_shut_down</span><span style="color: #000000;">.clear()
</span><span style="color: #0000ff;">try</span><span style="color: #000000;">:
</span><span style="color: #0000ff;">while</span> <span style="color: #0000ff;">not</span> self.<span style="color: #800080;">__shutdown_request</span><span style="color: #000000;">:
</span><span style="color: #008000;">#</span><span style="color: #008000;"> XXX: Consider using another file descriptor or</span>
<span style="color: #008000;">#</span><span style="color: #008000;"> connecting to the socket to wake this up instead of</span>
<span style="color: #008000;">#</span><span style="color: #008000;"> polling. Polling reduces our responsiveness to a</span>
<span style="color: #008000;">#</span><span style="color: #008000;"> shutdown request and wastes cpu at all other times.</span>
r,w,e =<span style="color: #000000;"> _eintr_retry(select.select,[self],[],poll_interval)
</span><span style="color: #0000ff;">if</span> self <span style="color: #0000ff;">in</span><span style="color: #000000;"> r:
self._handle_request_noblock()
</span><span style="color: #0000ff;">finally</span><span style="color: #000000;">:
self.</span><span style="color: #800080;">__shutdown_request</span> =<span style="color: #000000;"> False
self.</span><span style="color: #800080;">__is_shut_down</span><span style="color: #000000;">.set()
</span><span style="color: #0000ff;">def</span><span style="color: #000000;"> shutdown(self):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Stops the serve_forever loop.
Blocks until the loop has finished. This must be called while
serve_forever() is running in another thread,or it will
deadlock.
</span><span style="color: #800000;">"""</span><span style="color: #000000;">
self.</span><span style="color: #800080;">__shutdown_request</span> =<span style="color: #000000;"> True
self.</span><span style="color: #800080;">__is_shut_down</span><span style="color: #000000;">.wait()
</span><span style="color: #008000;">#</span><span style="color: #008000;"> The distinction between handling,getting,processing and</span>
<span style="color: #008000;">#</span><span style="color: #008000;"> finishing a request is fairly arbitrary. Remember:</span>
<span style="color: #008000;">#
<span style="color: #008000;">#<span style="color: #008000;"> - handle_request() is the top-level call. It calls<span style="color: #008000;">#<span style="color: #008000;"> select,get_request(),verify_request() and process_request()
<span style="color: #008000;">#<span style="color: #008000;"> - get_request() is different for stream or datagram sockets
<span style="color: #008000;">#<span style="color: #008000;"> - process_request() is the place that may fork a new process
<span style="color: #008000;">#<span style="color: #008000;"> or create a new thread to finish the request
<span style="color: #008000;">#<span style="color: #008000;"> - finish_request() instantiates the request handler class;
<span style="color: #008000;">#<span style="color: #008000;"> this constructor will handle the request all by itself
<span style="color: #0000ff;">def</span><span style="color: #000000;"> handle_request(self):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Handle one request,possibly blocking.
Respects self.timeout.
</span><span style="color: #800000;">"""</span>
<span style="color: #008000;">#</span><span style="color: #008000;"> Support people who used socket.settimeout() to escape</span>
<span style="color: #008000;">#</span><span style="color: #008000;"> handle_request before self.timeout was available.</span>
timeout =<span style="color: #000000;"> self.socket.gettimeout()
</span><span style="color: #0000ff;">if</span> timeout <span style="color: #0000ff;">is</span><span style="color: #000000;"> None:
timeout </span>=<span style="color: #000000;"> self.timeout
</span><span style="color: #0000ff;">elif</span> self.timeout <span style="color: #0000ff;">is</span> <span style="color: #0000ff;">not</span><span style="color: #000000;"> None:
timeout </span>=<span style="color: #000000;"> min(timeout,self.timeout)
fd_sets </span>=<span style="color: #000000;"> _eintr_retry(select.select,timeout)
</span><span style="color: #0000ff;">if</span> <span style="color: #0000ff;">not</span><span style="color: #000000;"> fd_sets[0]:
self.handle_timeout()
</span><span style="color: #0000ff;">return</span><span style="color: #000000;">
self._handle_request_noblock()
</span><span style="color: #0000ff;">def</span><span style="color: #000000;"> _handle_request_noblock(self):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Handle one request,without blocking.
I assume that select.select has returned that the socket is
readable before this function was called,so there should be
no risk of blocking in get_request().
</span><span style="color: #800000;">"""</span>
<span style="color: #0000ff;">try</span><span style="color: #000000;">:
request,client_address </span>=<span style="color: #000000;"> self.get_request()
</span><span style="color: #0000ff;">except</span><span style="color: #000000;"> socket.error:
</span><span style="color: #0000ff;">return</span>
<span style="color: #0000ff;">if</span><span style="color: #000000;"> self.verify_request(request,client_address):
</span><span style="color: #0000ff;">try</span><span style="color: #000000;">:
self.process_request(request,client_address)
</span><span style="color: #0000ff;">except</span><span style="color: #000000;">:
self.handle_error(request,client_address)
self.shutdown_request(request)
</span><span style="color: #0000ff;">def</span><span style="color: #000000;"> handle_timeout(self):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Called if no new request arrives within self.timeout.
Overridden by ForkingMixIn.
</span><span style="color: #800000;">"""</span>
<span style="color: #0000ff;">pass</span>
<span style="color: #0000ff;">def</span><span style="color: #000000;"> verify_request(self,request,client_address):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Verify the request. May be overridden.
Return True if we should proceed with this request.
</span><span style="color: #800000;">"""</span>
<span style="color: #0000ff;">return</span><span style="color: #000000;"> True
</span><span style="color: #0000ff;">def</span><span style="color: #000000;"> process_request(self,client_address):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Call finish_request.
Overridden by ForkingMixIn and ThreadingMixIn.
</span><span style="color: #800000;">"""</span><span style="color: #000000;">
self.finish_request(request,client_address)
self.shutdown_request(request)
</span><span style="color: #0000ff;">def</span><span style="color: #000000;"> server_close(self):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Called to clean-up the server.
May be overridden.
</span><span style="color: #800000;">"""</span>
<span style="color: #0000ff;">pass</span>
<span style="color: #0000ff;">def</span><span style="color: #000000;"> finish_request(self,client_address):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Finish one request by instantiating RequestHandlerClass.</span><span style="color: #800000;">"""</span><span style="color: #000000;">
self.RequestHandlerClass(request,client_address,self)
</span><span style="color: #0000ff;">def</span><span style="color: #000000;"> shutdown_request(self,request):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Called to shutdown and close an individual request.</span><span style="color: #800000;">"""</span><span style="color: #000000;">
self.close_request(request)
</span><span style="color: #0000ff;">def</span><span style="color: #000000;"> close_request(self,request):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Called to clean up an individual request.</span><span style="color: #800000;">"""</span>
<span style="color: #0000ff;">pass</span>
<span style="color: #0000ff;">def</span><span style="color: #000000;"> handle_error(self,client_address):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Handle an error gracefully. May be overridden.
The default is to print a traceback and continue.
</span><span style="color: #800000;">"""</span>
<span style="color: #0000ff;">print</span> <span style="color: #800000;">'</span><span style="color: #800000;">-</span><span style="color: #800000;">'</span>*40
<span style="color: #0000ff;">print</span> <span style="color: #800000;">'</span><span style="color: #800000;">Exception happened during processing of request from</span><span style="color: #800000;">'</span><span style="color: #000000;">,</span><span style="color: #0000ff;">print</span><span style="color: #000000;"> client_address
</span><span style="color: #0000ff;">import</span><span style="color: #000000;"> traceback
traceback.print_exc() </span><span style="color: #008000;">#</span><span style="color: #008000;"> XXX But this goes to stderr!</span>
<span style="color: #0000ff;">print</span> <span style="color: #800000;">'</span><span style="color: #800000;">-</span><span style="color: #800000;">'</span>*40</pre>
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Base class for various socket-based server classes.
Defaults to synchronous IP stream (i.e.,TCP).
Methods for the caller:
- __init__(server_address,RequestHandlerClass,bind_and_activate=True)
- serve_forever(poll_interval=0.5)
- shutdown()
- handle_request() # if you don't use serve_forever()
- fileno() -> int # for select()
Methods that may be overridden:
- server_bind()
- server_activate()
- get_request() -> request,client_address)
- process_request(request,client_address)
Class variables that may be overridden by derived classes or
instances:
- timeout
- address_family
- socket_type
- request_queue_size (only for stream sockets)
- allow_reuse_address
Instance variables:
- server_address
- RequestHandlerClass
- socket
</span><span style="color: #800000;">"""</span><span style="color: #000000;">
address_family </span>=<span style="color: #000000;"> socket.AF_INET
socket_type </span>=<span style="color: #000000;"> socket.SOCK_STREAM
request_queue_size </span>= 5<span style="color: #000000;">
allow_reuse_address </span>=<span style="color: #000000;"> False
</span><span style="color: #0000ff;">def</span> <span style="color: #800080;">__init__</span>(self,bind_and_activate=<span style="color: #000000;">True):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Constructor. May be extended,do not override.</span><span style="color: #800000;">"""</span><span style="color: #000000;">
BaseServer.</span><span style="color: #800080;">__init__</span><span style="color: #000000;">(self,RequestHandlerClass)
self.socket </span>=<span style="color: #000000;"> socket.socket(self.address_family,self.socket_type)
</span><span style="color: #0000ff;">if</span><span style="color: #000000;"> bind_and_activate:
</span><span style="color: #0000ff;">try</span><span style="color: #000000;">:
self.server_bind()
self.server_activate()
</span><span style="color: #0000ff;">except</span><span style="color: #000000;">:
self.server_close()
</span><span style="color: #0000ff;">raise</span>
<span style="color: #0000ff;">def</span><span style="color: #000000;"> server_bind(self):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Called by constructor to bind the socket.
May be overridden.
</span><span style="color: #800000;">"""</span>
<span style="color: #0000ff;">if</span><span style="color: #000000;"> self.allow_reuse_address:
self.socket.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,</span>1<span style="color: #000000;">)
self.socket.bind(self.server_address)
self.server_address </span>=<span style="color: #000000;"> self.socket.getsockname()
</span><span style="color: #0000ff;">def</span><span style="color: #000000;"> server_activate(self):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Called by constructor to activate the server.
May be overridden.
</span><span style="color: #800000;">"""</span><span style="color: #000000;">
self.socket.listen(self.request_queue_size)
</span><span style="color: #0000ff;">def</span><span style="color: #000000;"> server_close(self):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Called to clean-up the server.
May be overridden.
</span><span style="color: #800000;">"""</span><span style="color: #000000;">
self.socket.close()
</span><span style="color: #0000ff;">def</span><span style="color: #000000;"> fileno(self):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Return socket file number.
Interface required by select().
</span><span style="color: #800000;">"""</span>
<span style="color: #0000ff;">return</span><span style="color: #000000;"> self.socket.fileno()
</span><span style="color: #0000ff;">def</span><span style="color: #000000;"> get_request(self):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Get the request and client address from the socket.
May be overridden.
</span><span style="color: #800000;">"""</span>
<span style="color: #0000ff;">return</span><span style="color: #000000;"> self.socket.accept()
</span><span style="color: #0000ff;">def</span><span style="color: #000000;"> shutdown_request(self,request):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Called to shutdown and close an individual request.</span><span style="color: #800000;">"""</span>
<span style="color: #0000ff;">try</span><span style="color: #000000;">:
</span><span style="color: #008000;">#</span><span style="color: #008000;">explicitly shutdown. socket.close() merely releases</span>
<span style="color: #008000;">#</span><span style="color: #008000;">the socket and waits for GC to perform the actual close.</span>
<span style="color: #000000;"> request.shutdown(socket.SHUT_WR)
<span style="color: #0000ff;">pass <span style="color: #008000;">#<span style="color: #008000;">some platforms may raise ENOTCONN here
<span style="color: #000000;"> self.close_request(request)
</span><span style="color: #0000ff;">def</span><span style="color: #000000;"> close_request(self,request):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Called to clean up an individual request.</span><span style="color: #800000;">"""</span><span style="color: #000000;">
request.close()
TCPServer
<span style="color: #008000;">#</span><span style="color: #008000;"> Decides how threads will act upon termination of the</span>
<span style="color: #008000;">#</span><span style="color: #008000;"> main process</span>
daemon_threads =<span style="color: #000000;"> False
</span><span style="color: #0000ff;">def</span><span style="color: #000000;"> process_request_thread(self,client_address):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Same as in BaseServer but as a thread.
In addition,exception handling is done here.
</span><span style="color: #800000;">"""</span>
<span style="color: #0000ff;">try</span><span style="color: #000000;">:
self.finish_request(request,client_address)
self.shutdown_request(request)
</span><span style="color: #0000ff;">except</span><span style="color: #000000;">:
self.handle_error(request,client_address)
self.shutdown_request(request)
</span><span style="color: #0000ff;">def</span><span style="color: #000000;"> process_request(self,client_address):
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Start a new thread to process the request.</span><span style="color: #800000;">"""</span><span style="color: #000000;">
t </span>= threading.Thread(target =<span style="color: #000000;"> self.process_request_thread,args </span>=<span style="color: #000000;"> (request,client_address))
t.daemon </span>=<span style="color: #000000;"> self.daemon_threads
t.start()
ThreadingMixIn
</span><span style="color: #800000;">"""</span><span style="color: #800000;">Base class for request handler classes.
This class is instantiated for each request to be handled. The
constructor sets the instance variables request,client_address
and server,and then calls the handle() method. To implement a
specific service,all you need to do is to derive a class which
defines a handle() method.
The handle() method can find the request as self.request,the
client address as self.client_address,and the server (in case it
needs access to per-server information) as self.server. Since a
separate instance is created for each request,the handle() method
can define arbitrary other instance variariables.
</span><span style="color: #800000;">"""</span>
<span style="color: #0000ff;">def</span> <span style="color: #800080;">__init__</span><span style="color: #000000;">(self,server):
self.request </span>=<span style="color: #000000;"> request
self.client_address </span>=<span style="color: #000000;"> client_address
self.server </span>=<span style="color: #000000;"> server
self.setup()
</span><span style="color: #0000ff;">try</span><span style="color: #000000;">:
self.handle()
</span><span style="color: #0000ff;">finally</span><span style="color: #000000;">:
self.finish()
</span><span style="color: #0000ff;">def</span><span style="color: #000000;"> setup(self):
</span><span style="color: #0000ff;">pass</span>
<span style="color: #0000ff;">def</span><span style="color: #000000;"> handle(self):
</span><span style="color: #0000ff;">pass</span>
<span style="color: #0000ff;">def</span><span style="color: #000000;"> finish(self):
</span><span style="color: #0000ff;">pass</span><span style="color: #000000;">
SocketServer.BaseRequestHandler
?SocketServer的ThreadingTCPServer之所以可以同时处理请求得益于?select?和?Threading?两个模块,其实本质上就是在服务器端为每一个客户端创建一个线程,当前线程用来处理对应客户端的请求,所以,可以支持同时多个个客户端连接。
ForkingTCPServer
ForkingTCPServer和ThreadingTCPServer的使用和执行流程基本一致,只不过在内部分别为请求者建立 “线程” ?和 “进程”。
(编辑:李大同)
【声明】本站内容均来自网络,其相关言论仅代表作者个人观点,不代表本站立场。若无意侵犯到您的权利,请及时与联系站长删除相关内容!