openstack swift和wsgi源码分析2 eventlet HTTP协议解析过程中rf

研究rfile如何解析消息头和消息正文，并返回结果的过程。

rfile和wfile 根据 client_socket 由HttpProtocal 类的setup函数生成。

生成流程，如下代码：

def server():

client_socket = sock.accept()
pool.spawn_n(serv.process_request,client_socket)

def process_request(self,sock_params):

sock,address = sock_params
proto.__init__(sock,address,self)

class BaseRequestHandler:

def __init__(self,request,client_address,server):
    self.request = request
    self.client_address = client_address
    self.server = server

class HttpProtocol

def setup(self):
    conn = self.connection = self.request
    try:
        self.rfile = conn.makefile('rb',self.rbufsize)
        self.wfile = conn.makefile('wb',self.wbufsize)
    except (AttributeError,NotImplementedError):
        if hasattr(conn,'send') and hasattr(conn,'recv'):
            # it's an SSL.Connection
            self.rfile = socket._fileobject(conn,"rb",self.rbufsize)
            self.wfile = socket._fileobject(conn,"wb",self.wbufsize)

其中sock 由 eventlet 所重写的listen函数生成，基于eventlet.greenio模块的GreenSocket 类。

self.connection调用makefile创建一个与该套接字相关连的文件

通过调试 self.rfile.readline函数(位于socket模块)，可以判断python对于http的协议主要是根据数据中的/r/n分隔符，区分开消息头和消息正文,且消息正文长度由content-length消息头决定.

客户端发送数据：

POST /oauth/access_token HTTP/1.1
User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7
NSS/3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
Host: localhost
Accept: /
Content-Length: 51
Content-Type: application/x-www-form-urlencoded

函数入下：

def readline(self,size=-1):
    buf = self._rbuf
    buf.seek(0,2)  # seek end
    if buf.tell() > 0:
        # check if we already have it in our buffer
        buf.seek(0)
        bline = buf.readline(size)
        if bline.endswith('n') or len(bline) == size:
            self._rbuf = StringIO()
            self._rbuf.write(buf.read())
            return bline
        del bline
    if size < 0:
        # Read until n or EOF,whichever comes first
        if self._rbufsize <= 1:
            # Speed up unbuffered case
            buf.seek(0)
            buffers = [buf.read()]
            self._rbuf = StringIO()  
            data = None
            recv = self._sock.recv
            while True:
                try:
                    while data != "n":
                        data = recv(1)
                        if not data:
                            break
                        buffers.append(data)
                except error,e:
                    # The try..except to catch EINTR was moved outside the
                    # recv loop to avoid the per byte overhead.
                    if _exception_was_EINTR(e):
                        continue
                    raise
                break
            return "".join(buffers)


        buf.seek(0,2)  # seek end
        self._rbuf = StringIO()  # reset _rbuf.  we consume it via buf.
        while True:
            try:
                data = self._sock.recv(self._rbufsize)
            except error,e:
                if _exception_was_EINTR(e):
                    continue
                raise
            if not data:
                break
            nl = data.find('n')
            if nl >= 0:
                nl += 1
                buf.write(data[:nl])
                self._rbuf.write(data[nl:])
                del data
                break
            buf.write(data)
        return buf.getvalue()
    else:
        # Read until size bytes or n or EOF seen,whichever comes first
        buf.seek(0,2)  # seek end
        buf_len = buf.tell()
        if buf_len >= size:
            buf.seek(0)
            rv = buf.read(size)
            self._rbuf = StringIO()
            self._rbuf.write(buf.read())
            return rv
        self._rbuf = StringIO()  # reset _rbuf.  we consume it via buf.
        while True:
            try:
                data = self._sock.recv(self._rbufsize)
            except error,e:
                if _exception_was_EINTR(e):
                    continue
                raise
            if not data:
                break
            left = size - buf_len
            # did we just receive a newline?
            nl = data.find('n',left)
            if nl >= 0:
                nl += 1
                # save the excess data to _rbuf
                self._rbuf.write(data[nl:])
                if buf_len:
                    buf.write(data[:nl])
                    break
                else:
                    # Shortcut.  Avoid data copy through buf when
                    # returning
                    # a substring of our first recv().
                    return data[:nl]
            n = len(data)
            if n == size and not buf_len:
                # Shortcut.  Avoid data copy through buf when
                # returning exactly all of our first recv().
                return data
            if n >= left:
                buf.write(data[:left])
                self._rbuf.write(data[left:])
                break
            buf.write(data)
            buf_len += n
            #assert buf_len == buf.tell()
        return buf.getvalue()

注：如果消息正文Transfer-Encoding:chunked 方式传输，则不实用上述解析方式。

最终消息头数据存储在self.environ变量中，消息正文由env[‘eventlet.input’]保存。

其中env[‘eventlet.input’] 的创建方式如下

env[‘wsgi.input’] =Input(self.rfile,length,
wfile=wfile,wfile_line=wfile_line,chunked_input=chunked)

若self.rfile 所读取的数据为”,则表明收到客户端答复，释放网络资源。