在传输层上压缩WebService的请求和响应

发布时间：2020-12-16 23:33:47 所属栏目：安全来源：网络整理

导读：http://www.blogjava.net/mstar/archive/2013/06/23/400884.html 场景 ? 场景是这样的：客户端.NET 3.5应用程序，WCF实现WebService调用，服务端Java，通过CXF提供WebService。有一个方法提供了有一个字符串类型的参数，实际生产环境里会传100k以上的字符

http://www.blogjava.net/mstar/archive/2013/06/23/400884.html

场景?

场景是这样的：客户端.NET 3.5应用程序，WCF实现WebService调用，服务端Java，通过CXF提供WebService。有一个方法提供了有一个字符串类型的参数，实际生产环境里会传100k以上的字符串。在并发量比较大的情况下，带宽占用很严重。所以寻找一种可以把传输的 SOAP消息在客户端压缩，服务端解压缩的方法。?

这里提供的方式在是客户端通过WCF的MessageEncoder机制对所有的SOAP请求消息压缩，SOAP响应消息解压缩，反过来在服务端通过一个Filter对所有的SOAP请求消息，对SOAP响应消息压缩。?
请求的流程如下：
Client -> SOAP Request -> GzipMessageEncoder -> gzip binary -> GzipWebSericeFilter -> SOAP Request -> CXF?
响应的流程如下：
CXF -> SOAP Response -> GzipWebServiceFilter -> gzip binary -> GzipMessageEncoder -> SOAP Response -> Client?
其中.NET的WCF的GzipMessageEncoder是参照 WCF的Samples ，下载解压后路径WF_WCF_SamplesWCFExtensibilityMessageEncoderCompression
客户端
下面先来看一下客户端部分的代码：?
GZipMessageEncoderFactory.cs 这文件主要是提供GZipMessageEncoder，在里面通过重写ReadMessage和WriteMessage方法来实现压缩和解压缩。实际压缩和解压处理是使用GZipStream实现的。?

namespace ?ConsoleApplication2
{
???? // This?class?is?used?to?create?the?custom?encoder?(GZipMessageEncoder)
???? internal ? class ?GZipMessageEncoderFactory?:?MessageEncoderFactory
????{
???????? readonly ?MessageEncoder?_encoder;

???????? The?GZip?encoder?wraps?an?inner?encoder
???????? We?require?a?factory?to?be?passed?in?that?will?create?this?inner?encoder ???????? public ?GZipMessageEncoderFactory(MessageEncoderFactory?messageEncoderFactory)
????????{
???????????? if ?(messageEncoderFactory? == null )
???????????????? throw new ?ArgumentNullException( " messageEncoderFactory ,? A?valid?message?encoder?factory?must?be?passed?to?the?GZipEncoder );
????????????_encoder? = ?GZipMessageEncoder(messageEncoderFactory.Encoder);

????????}
????????
???????? The?service?framework?uses?this?property?to?obtain?an?encoder?from?this?encoder?factory override ?MessageEncoder?Encoder
????????{
???????????? get ?{? return ?_encoder;?}
????????}

???????? ?MessageVersion?MessageVersion
????????{
???????????? ?_encoder.MessageVersion;?}
????????}

???????? This?is?the?actual?GZip?encoder ?GZipMessageEncoder?:?MessageEncoder
????????{
???????????? private const string ?GZipMediaType? application/x-gzip ;
???????????? ?GZipContentType? + ;?charset=utf-8 ;

???????????? This?implementation?wraps?an?inner?encoder?that?actually?converts?a?WCF?Message
???????????? into?textual?XML,?binary?XML?or?some?other?format.?This?implementation?then?compresses?the?results.
???????????? The?opposite?happens?when?reading?messages.
???????????? This?member?stores?this?inner?encoder. ???????????? ?MessageEncoder?_innerEncoder;

???????????? We?require?an?inner?encoder?to?be?supplied?(see?comment?above) ?GZipMessageEncoder(MessageEncoder?messageEncoder)
????????????{
???????????????? ?(messageEncoder? )
???????????????????? messageEncoder A?valid?message?encoder?must?be?passed?to?the?GZipEncoder );
????????????????_innerEncoder? ?messageEncoder;
????????????}

???????????? ?ContentType
????????????{
???????????????? ?GZipContentType;?}
????????????}

???????????? ?MediaType
????????????{
???????????????? ?GZipMediaType;?}
????????????}

???????????? SOAP?version?to?use?-?we?delegate?to?the?inner?encoder?for?this ?MessageVersion?MessageVersion
????????????{
???????????????? ?_innerEncoder.MessageVersion;?}
????????????}

???????????? bool ?IsContentTypeSupported( ?contentType)
????????????{
???????????????? ?contentType.StartsWith(GZipMediaType,?StringComparison.OrdinalIgnoreCase)? || ?contentType.StartsWith( text/xml Helper?method?to?compress?an?array?of?bytes static ?ArraySegment < byte > ?CompressBuffer(ArraySegment ?buffer,?BufferManager?bufferManager,255)">int ?messageOffset)
????????????{
????????????????var?memoryStream? ?MemoryStream();
????????????????memoryStream.Write(buffer.Array,0)">0 using ?(var?gzStream? ?GZipStream(memoryStream,?CompressionMode.Compress,255)">true ))
????????????????{
????????????????????gzStream.Write(buffer.Array,?messageOffset,?buffer.Count);
????????????????}

????????????????var?compressedBytes? ?memoryStream.ToArray();
????????????????var?bufferedBytes? ?bufferManager.TakeBuffer(compressedBytes.Length);

????????????????Array.Copy(compressedBytes,?bufferedBytes,?compressedBytes.Length);

????????????????bufferManager.ReturnBuffer(buffer.Array);
????????????????var?byteArray? (bufferedBytes,?bufferedBytes.Length? - ?messageOffset);

???????????????? ?byteArray;
????????????}

???????????? Helper?method?to?decompress?an?array?of?bytes ?DecompressBuffer(ArraySegment ?MemoryStream(buffer.Array,?buffer.Offset,?buffer.Count? ?buffer.Offset);
????????????????var?decompressedStream? ?MemoryStream();
???????????????? ?blockSize? 1024 ;
???????????????? []?tempBuffer? ?bufferManager.TakeBuffer(blockSize);
???????????????? while ?( )
????????????????????{
????????????????????????var?bytesRead? ?gzStream.Read(tempBuffer,?blockSize);
???????????????????????? ?(bytesRead? )
???????????????????????????? break ;
????????????????????????decompressedStream.Write(tempBuffer,?bytesRead);
????????????????????}
????????????????}
????????????????bufferManager.ReturnBuffer(tempBuffer);

????????????????var?decompressedBytes? ?decompressedStream.ToArray();
????????????????var?bufferManagerBuffer? ?bufferManager.TakeBuffer(decompressedBytes.Length? ?buffer.Offset);
????????????????Array.Copy(buffer.Array,?bufferManagerBuffer,?buffer.Offset);
????????????????Array.Copy(decompressedBytes,?decompressedBytes.Length);

????????????????var?byteArray? (bufferManagerBuffer,?decompressedBytes.Length);
????????????????bufferManager.ReturnBuffer(buffer.Array);

???????????????? One?of?the?two?main?entry?points?into?the?encoder.?Called?by?WCF?to?encode?a?Message?into?a?buffered?byte?array. ?WriteMessage(Message?message,0)">?maxMessageSize,0)">?messageOffset)
????????????{
???????????????? Use?the?inner?encoder?to?encode?a?Message?into?a?buffered?byte?array ????????????????ArraySegment ?buffer? ?_innerEncoder.WriteMessage(message,?maxMessageSize,?bufferManager,?messageOffset);
???????????????? Compress?the?resulting?byte?array ???????????????? ?CompressBuffer(buffer,?messageOffset);
????????????}

???????????? ?Message?ReadMessage(Stream?stream,0)">?maxSizeOfHeaders,0)">?contentType)
????????????{
????????????????var?gzStream? ?GZipStream(stream,?CompressionMode.Decompress,0)">);
???????????????? ?_innerEncoder.ReadMessage(gzStream,?maxSizeOfHeaders);
????????????}

???????????? ?Message?ReadMessage(ArraySegment Decompress?the?buffer ?decompressedBuffer? ?DecompressBuffer(buffer,?bufferManager);
???????????????? Use?the?inner?encoder?to?decode?the?decompressed?buffer ????????????????Message?returnMessage? ?_innerEncoder.ReadMessage(decompressedBuffer,?bufferManager);
????????????????returnMessage.Properties.Encoder? this ?returnMessage;
????????????}

???????????? void ))
????????????????{
????????????????????_innerEncoder.WriteMessage(message,?gzStream);
????????????????}

???????????????? ?innerEncoder.WriteMessage(message,?gzStream)?depends?on?that?it?can?flush?data?by?flushing?
???????????????? ?the?stream?passed?in,?but?the?implementation?of?GZipStream.Flush?will?not?flush?underlying
???????????????? ?stream,?so?we?need?to?flush?here. ????????????????stream.Flush();
????????????}
????????}
????}
}

下面是GZipMessageEncodingBindingElement.cs 这里的GZipMessageEncodingBindingElement类是为了在app.config里添加配置项。?
This?is?the?binding?element?that,?when?plugged?into?a?custom?binding,?will?enable?the?GZip?encoder sealed ?GZipMessageEncodingBindingElement?
????????????????????????:?MessageEncodingBindingElement? BindingElement ????{

???????? We?will?use?an?inner?binding?element?to?store?information?required?for?the?inner?encoder ????????MessageEncodingBindingElement?_innerBindingElement;

???????? By?default,?use?the?default?text?encoder?as?the?inner?encoder ?GZipMessageEncodingBindingElement()
????????????:? ( ?TextMessageEncodingBindingElement())?{?}

???????? ?GZipMessageEncodingBindingElement(MessageEncodingBindingElement?messageEncoderBindingElement)
????????{
????????????_innerBindingElement? ?messageEncoderBindingElement;
????????}

???????? ?MessageEncodingBindingElement?InnerMessageEncodingBindingElement
????????{
???????????? ?_innerBindingElement;?}
???????????? set ?{?_innerBindingElement? ?value;?}
????????}

???????? Main?entry?point?into?the?encoder?binding?element.?Called?by?WCF?to?get?the?factory?that?will?create?the
???????? message?encoder ?MessageEncoderFactory?CreateMessageEncoderFactory()
????????{
???????????? ?GZipMessageEncoderFactory(_innerBindingElement.CreateMessageEncoderFactory());
????????}
???????
???????? ?_innerBindingElement.MessageVersion;?}
???????????? ?{?_innerBindingElement.MessageVersion? ?BindingElement?Clone()
????????{
???????????? ?GZipMessageEncodingBindingElement(_innerBindingElement);
????????}

???????? ?T?GetProperty T (BindingContext?context)
????????{
???????????? typeof (T)? (XmlDictionaryReaderQuotas))
????????????{
???????????????? ?_innerBindingElement.GetProperty (context);
????????????}
???????????? base .GetProperty (context);
????????}

???????? ?IChannelFactory TChannel ?BuildChannelFactory ?(context? context );

????????????context.BindingParameters.Add( );
???????????? ?context.BuildInnerChannelFactory ();
????????}

???????? ?IChannelListener ?BuildChannelListener ?context.BuildInnerChannelListener ?CanBuildChannelListener ?context.CanBuildInnerChannelListener ();
????????}
????}

???? This?class?is?necessary?to?be?able?to?plug?in?the?GZip?encoder?binding?element?through
???? a?configuration?file ?GZipMessageEncodingElement?:?BindingElementExtensionElement
????{
???????? Called?by?the?WCF?to?discover?the?type?of?binding?element?this?config?section?enables ?Type?BindingElementType
????????{
???????????? (GZipMessageEncodingBindingElement);?}
????????}

???????? The?only?property?we?need?to?configure?for?our?binding?element?is?the?type?of
???????? inner?encoder?to?use.?Here,?we?support?text?and?binary. ????????[ConfigurationProperty( innerMessageEncoding textMessageEncoding )]
???????? ?InnerMessageEncoding
????????{
???????????? ) [ ];?}
???????????? ]? messageVersion Soap12 ?MessageVersion
????????{
???????????? Called?by?the?WCF?to?apply?the?configuration?settings?(the?property?above)?to?the?binding?element ?ApplyConfiguration(BindingElement?bindingElement)
????????{
????????????var?binding? ?(GZipMessageEncodingBindingElement)bindingElement;
????????????PropertyInformationCollection?propertyInfo? ?ElementInformation.Properties;
????????????var?propertyInformation? ?propertyInfo[ ];
???????????? ?(propertyInformation? ?propertyInformation.ValueOrigin? ?PropertyValueOrigin.Default)? ;

????????????var?version? ?System.ServiceModel.Channels.MessageVersion.Soap12;
???????????? Soap11 ?MessageVersion)
????????????{
????????????????version? ?System.ServiceModel.Channels.MessageVersion.Soap11;
????????????}

???????????? switch ?(InnerMessageEncoding)
????????????{
???????????????? case :
????????????????????binding.InnerMessageEncodingBindingElement? ?TextMessageEncodingBindingElement()?{?MessageVersion? ?version?};
???????????????????? binaryMessageEncoding ?BinaryMessageEncodingBindingElement();
???????????????????? ;
????????????}
????????}

???????? Called?by?the?WCF?to?create?the?binding?element protected ?BindingElement?CreateBindingElement()
????????{
????????????var?bindingElement? ?GZipMessageEncodingBindingElement();
????????????ApplyConfiguration(bindingElement);
???????????? ?bindingElement;
????????}
????}
}

然后我们就可以把这个GZipMessageEncodingElement配置到app.config里了?
<? xml?version="1.0"?encoding="utf-8"? ?> < configuration >
?? system .serviceModel
???? extensions
?????? bindingElementExtensions
???????? add? name ="gzipMessageEncoding" ?type ="ConsoleApplication2.GZipMessageEncodingElement,ConsoleApplication2" /> </ bindings customBinding binding? ="countServiceSoapBinding"
?????????? gzipMessageEncoding? ="textMessageEncoding" ?messageVersion ="Soap11" httpTransport? manualAddressing ="false"
?????????????????????????authenticationScheme ="Anonymous"
?????????????????????????bypassProxyOnLocal
?????????????????????????hostNameComparisonMode ="StrongWildcard"
?????????????????????????proxyAuthenticationScheme
?????????????????????????realm =""
?????????????????????????useDefaultWebProxy ="true" binding client endpoint? address ="http://192.168.2.3:8080/binder/services/countService"
??????????binding ="customBinding" ?bindingConfiguration ="countServiceSoapBinding"
??????????contract ="ServiceReference1.HolidayService" ?name ="HolidayServiceImplPort" system.serviceModel

客户端最后的部分就是调用webservice，这里的压缩和解压对于调用者和陪调用者是透明的。也就是同没有压缩和解压之前的使用方法一样。?
?Program
????{
???????? ?Main( []?args)
????????{
???????????? try
????????????{
????????????????var?service? ?ServiceReference1.HolidayServiceClient();
????????????????var?text? File.ReadAllText( c:words );

????????????????var?len? ?service.countText(text);
????????????????Console.WriteLine( lenght?=?{0} catch ?(Exception?e)
????????????{
????????????????Console.WriteLine(e.Message);
????????????????Console.WriteLine(e.StackTrace);
????????????}
????????????Console.Read();
????????}
????}
}

服务端
服务端是一个Filter，和HttpServletRequest和HttpServletResponse的包装类。?
入口：GzipWebServiceFilter.java?
/**
?*?把使用Gzip压缩的SOAP消息解压缩。
?*? @author ?matianyi
?*
? */ ?GzipWebServiceFilter? implements ?Filter?{

???? final ?String?CONTENT_TYPE? ;
???? ?String?CONTENT_ENCODING? utf-8 ;

????@Override
???? ?init(FilterConfig?filterConfig)? throws ?ServletException?{
???????? ?TODO?Auto-generated?method?stub
????}

????@SuppressWarnings( unchecked )
????@Override
???? ?doFilter(ServletRequest?request,?ServletResponse?response,
????????????FilterChain?chain)? ?IOException,?ServletException?{
????????
????????HttpServletRequest?req? ?(HttpServletRequest)?request;
????????HttpServletResponse?resp? ?(HttpServletResponse)?response;
????????
???????? (req.getContentType()? ! req.getContentType().startsWith(CONTENT_TYPE)){
????????????chain.doFilter(request,?response);
????????}? else ?{
????????????chain.doFilter( ?GzipHttpServletRequestWrapper(req),0)">?GzipHttpServletResponseWrapper(resp));
????????}
????}

????@Override
???? ?destroy()?{
????????
????}

}
这里就是判断contentType，如果是gzip的就用GzipHttpServletRequestWrapper和GzipHttpServletResponseWrapper包装原始的Request和Response以实现压缩和解压缩。?
GzipHttpServletRequestWrapper?
?GzipHttpServletRequestWrapper? extends ?HttpServletRequestWrapper?{

???? ?String?CONTNET_TYPE_SOAP_1_2? application/soap+xml ?String?CONTNET_TYPE_SOAP_1_1? ;

???? ?GzipHttpServletRequestWrapper(HttpServletRequest?request)?{
???????? super (request);
????}

????@Override
???? ?ServletInputStream?getInputStream()? ?IOException?{
???????? ?GzipServletInputStream( .getInputStream());
????}

????@Override
???? ?String?getContentType()?{
???????? ?CONTNET_TYPE_SOAP_1_2;
????}

????@Override
???? ?String?getHeader(String?name)?{
???????? content-type .equalsIgnoreCase(name))?{
???????????? ?getContentType();
????????}? ?{
???????????? .getHeader(name);
????????}
????}

}

?GzipServletInputStream? ?ServletInputStream?{

???? ?GZIPInputStream?delegate;

???? ?GzipServletInputStream(ServletInputStream?servletInputStream)
???????????? ();
???????? .delegate? ?GZIPInputStream(servletInputStream);
????}

????@Override
???? ?read()? ?delegate.read();
????}

}
GzipHttpServletResponseWrapper?
?GzipHttpServletResponseWrapper? ?HttpServletResponseWrapper?{

???? ?GzipHttpServletResponseWrapper(HttpServletResponse?response)?{
???????? (response);
????}

????@Override
???? ?ServletOutputStream?getOutputStream()? ?GzipServletOutputStream( .getOutputStream());
????}

????@Override
???? ?setCharacterEncoding(String?charset)?{
???????? .setCharacterEncoding(GzipWebServiceFilter.CONTENT_ENCODING);
????}

????@Override
???? ?setContentType(String?type)?{
???????? .setContentType(GzipWebServiceFilter.CONTENT_TYPE? ;?charset= ?GzipWebServiceFilter.CONTENT_ENCODING);
????}
????
}

?GzipServletOutputStream? ?ServletOutputStream{
???? ?GZIPOutputStream?delegate;

???? ?GzipServletOutputStream(ServletOutputStream?servletOutputStream)
???????????? ?GZIPOutputStream(servletOutputStream);
????}
????
????
????@Override
???? ?write( ?b)? ?IOException?{
????????System.out.print(( char )b);
????????delegate.write(b);
????}

???? ?close()? ?IOException?{
????????delegate.close();
????}

???? ?flush()? ?IOException?{
????????delegate.flush();
????}

???? []?buf,0)">?off,0)">?len)? ?IOException?{
????????delegate.write(buf,?off,?len);
????}

???? []?b)? ?IOException?{
????????delegate.write(b);
????}
????
????
} 这里做的主要事情就是在Resquest的getInputStream和Response的getOutputStream是返回一个拥有GZip功能的Stream，来代替原始的Stream。通过原始的Stream仍然是最终的输入和输出源。?
然后在web.xml中把这个Filter作用于原来的WebService的Servlet?
web.xml?
xml?version="1.0"?encoding="UTF-8" web-app? version ="2.5" ?xmlns ="http://java.sun.com/xml/ns/javaee"
????xmlns:xsi ="http://www.w3.org/2001/XMLSchema-instance"
????xsi:schemaLocation ="http://java.sun.com/xml/ns/javaee?http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"

????  context-param param-name contextConfigLocation param-value /WEB-INF/spring/root-context.xml
????
???? ?Creates?the?Spring?Container?shared?by?all?Servlets?and?Filters? listener listener-class org.springframework.web.context.ContextLoaderListener filter filter-name GzipWebServiceFilter filter-class com.cccis.ws.GzipWebServiceFilter filter-mapping url-pattern /services/* servlet description Apache?CXF?Endpoint servlet-name cxf servlet-class org.apache.cxf.transport.servlet.CXFServlet load-on-startup 1 servlet-mapping

web-app >
webservice的配置和cxf原来的一样?
beans? xmlns ="http://www.springframework.org/schema/beans"
????xmlns:context ="http://www.springframework.org/schema/context"
????xmlns:jaxws ="http://cxf.apache.org/jaxws" ="http://www.springframework.org/schema/beans?http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
???????http://www.springframework.org/schema/context?http://www.springframework.org/schema/context/spring-context.xsd
???????http://cxf.apache.org/jaxws?http://cxf.apache.org/schemas/jaxws.xsd" import? resource ="classpath:META-INF/cxf/cxf.xml" ="classpath:META-INF/cxf/cxf-servlet.xml" bean? id ="countServiceImpl" ?class ="com.cccis.ws.HolidayServiceImpl" jaxws:endpoint?
?????? ="countService" ?
??????implementor ="#countServiceImpl" ?
??????serviceName
??????address ="/countService"
??????
beans > 如果你想看一下实际的HTTP请求和响应是什么样子的可以用Fiddler Web Debugger来查看?

本文的源代码在附件中。?
本文的方案没有在最终的被用于生产环境，一个原因是比较复杂，另外一个是服务器在对大XML进行unmarshal的效率并不高。单本文的方案的好处就是不用对原有的webservice接口和实现进行修改。最后在实际场景用我们使用 MTOM 来解决问题的，后面我还会写一篇文章来介绍这个方法。?

（编辑：李大同）

【声明】本站内容均来自网络，其相关言论仅代表作者个人观点，不代表本站立场。若无意侵犯到您的权利，请及时与联系站长删除相关内容!