【原创】源码解析 packetbeat 之 responsetime 计算问题

发布时间：2020-12-16 18:16:28 所属栏目：大数据来源：网络整理

导读：问题使用 packetbeat 进行抓包分析的主要目的之一就是确定 request 和 response 之间 latency 时间；而 responsetime 值对应的就是这个时间，因此该值的计算方式，以及准确性在实际使用中非常重要；表现在基于 packetbeat 的分析报告中，经常可以看到如

问题

使用 packetbeat 进行抓包分析的主要目的之一就是确定 request 和 response 之间 latency 时间；而 responsetime 值对应的就是这个时间，因此该值的计算方式，以及准确性在实际使用中非常重要；

表现

在基于 packetbeat 的分析报告中，经常可以看到如下输出内容

responsetime(193 microseconds)	==>    No.<1>
----
{"@timestamp":"2017-03-29T08:52:01.965Z","beat":{"hostname":"sunfeideMacBook-Pro.local","name":"sunfeideMacBook-Pro.local","version":"6.0.0-alpha1"},"bytes_in":51,"bytes_out":4,"client_ip":"10.0.58.183","client_port":26776,"client_proc":"","client_server":"","ip":"10.0.10.58","method":"SMEMBERS","port":7602,"proc":"","query":"SMEMBERS restaurant:326200:contents","redis":{"return_value":"[]"},"resource":"restaurant:326200:contents","responsetime":193,"server":"","status":"OK","type":"redis"}


responsetime(193 microseconds)	==>    No.<2>
----
{"@timestamp":"2017-03-29T08:52:01.965Z","bytes_in":57,"query":"SMEMBERS food_restaurant:2186309:contents","resource":"food_restaurant:2186309:contents","type":"redis"}

从中可以看到几个数值相同的地方：

<1> 和 <2> 的 "@timestamp" 数值相同；
<1> 和 <2> 的 "responsetime" 数值相同；
<1> 和 <2> 的 "client_port" 和 "port" 相同；

不同点在于：

"query" 的内容不同；

看起来似乎是在同一个时间发出了不同 query 请求，但在获取相应的时间时，得到了相同的值；其实这个是由于 pipeline 的原因，如下图所示：

还存在另外一种情况：即 response 大到需要在 TCP 协议层面分包发送；

日志如下：

{"@timestamp":"2017-03-29T08:52:06.501Z","bytes_in":52,"bytes_out":13507,"method":"GET","query":"GET search_area_restaurant_id:wwmthp","redis":{"return_value":"(lp1nI500232naI1777849naI619754naI1403142naI523501naI2350975naI151675482naI1505143naI1069044naI614157naI1406283naI571267naI2069336naI151695752naI147163644naI2239901naI2256112naI451580naI1217250naI381736naI528265naI634537naI767398naI475421naI1505027naI400907naI152107900naI491118naI416144naI150970671naI484445naI150058323naI1048628naI317081naI552229naI519132naI1962390naI2262288naI150899176naI1509387naI1551358naI994255naI151004498naI324628naI871239naI2020650naI2059480naI921549naI343032naI2251490naI192074naI305984naI1333436naI1559413naI151691503naI578980naI2359311naI1069859naI551534naI688005naI1090091naI147502706naI2060663naI1189870naI1810063naI150162189naI1053551naI1881428naI1265721naI2349741naI429957naI401978naI2113721naI956012naI150969522naI216694naI752204naI1913280naI408229naI489046naI1973669naI150973128naI907450naI1858036naI2047470naI2310933naI1420537naI396459naI147508072naI314288naI1467056naI150897864naI1500140naI295317naI1260075naI202927naI150101723naI1051636naI901363naI1126673naI1148076naI959501naI421094naI453764naI1012326naI604775naI602103naI2132709naI150160637naI150851767naI85240naI525522naI1147925naI150883092naI1897103naI1048713naI456792naI2235222naI396450naI2224988naI150972019naI1519190naI1425052naI150855322naI151642336naI1915130naI2164310naI2174705naI729864naI671254naI1291063naI2028744naI1107294naI52027naI556973naI150992786naI200796naI1035913naI648504naI150052844naI1972231naI1904258naI1304882naI792645naI150136030naI2308724naI2308727naI648695naI951796naI753048naI1536919naI1469356naI497237naI2020502naI676226naI2195107naI952901naI2267803naI1024375naI2203394naI1321801naI1906374naI558568naI394262naI1988030naI1056855naI168082naI675100naI2247914naI800164naI944599naI1522307naI151694252naI652848naI1866111naI1866441naI852980naI1783720naI1148100naI283750naI1148102naI1001539naI1823360naI572147naI1119549naI384890naI434975naI2173503naI446636naI202961naI901370naI635638naI2174591naI2199680naI1385546naI1001902naI1900579naI497519naI1885574naI1028157naI1397383naI150879109naI223911naI1935449naI1147851naI376707naI151704231naI943972naI1974773naI1866233naI886223naI659015naI1860886naI1085306naI1050226naI143482230naI448164naI429877naI2367972naI147587877naI983749naI1181122naI1304339naI150967237naI626696naI479785naI1936405naI509876naI1524715naI151003408naI1872271naI2188925naI1337903naI1768761naI2140255naI142942425naI2055240naI1060960naI150135720naI777810naI570124naI2055160naI693420naI71235naI1785398naI1048273naI402562naI2026673naI660297naI2083100naI143688naI1435957naI2246665naI2061987naI1924181naI893305naI614226naI150879529naI151682687naI1393682naI630763naI797508naI1875434naI1838449naI694343naI1049983naI1863797naI1192097naI150895097naI685287naI381696naI147523922naI951867naI1476534naI1277845naI2147422naI150140415naI1908701naI1880458naI2330461naI1438015naI150068727naI1148535naI1371845naI2216946naI150162011naI1115466naI1824764naI1098816naI144028081naI1784923naI2231055naI2004869naI150893394naI2242511naI1050045naI150135634naI150039604naI1491841naI1469786naI143157238naI1418345naI151690609naI1148089naI662320naI1461795naI772502naI552200naI2238979naI150156167naI812167naI1781115naI706669naI2221035naI150093078naI620689naI732740naI713059naI1534419naI150852898naI525515naI643672naI707182naI1185116naI1276811naI143325050naI1423488naI1842657naI1053962naI329073naI777222naI1469234naI1547042naI150898598naI390550naI672301naI893121naI1322086naI461795naI948080naI497499naI729717naI662143naI1157222naI1306601naI1157224naI2256341naI283716naI1047991naI1289969naI150136007naI723735naI417641naI280016naI996864naI144036295naI1424001naI1398126naI1143989naI1524495naI565435naI1054112naI1767947naI1965142naI2160880naI341681naI1395098naI145354205naI679137naI2355156naI1350456naI552106naI478414naI150884935naI720549naI630986naI360173naI1800584naI1880122naI2004524naI150007275naI620424naI1350075naI150024231naI2132708naI867622naI150140660naI1420923naI1003837naI1964620naI1820812naI495095naI1181000naI800117naI602382naI221984naI925799naI152107221naI498411naI1325055naI550321naI2100070naI58897naI679554naI143845525naI1220700naI1847426naI1879887naI419552naI274958naI2003755naI1933054naI1055687naI629928naI638178naI2374015naI150160469naI1339305naI130659naI220632naI1052448naI471485naI150160459naI1885033naI1421051naI2195803naI150029456naI150092202naI637098naI1047981naI2162072naI464591naI143256668naI542556naI536273naI926974naI788329naI1881438naI494837naI1253964naI150989212naI451522naI1284806naI873353naI2054650naI2242292naI276741naI1417938naI1396276naI1443248naI150079414naI1066834naI683853naI626558naI457793naI428432naI1001827naI1464245naI298016naI380131naI2160210naI536458naI2174707naI2051611naI150966081naI1067887naI150067266naI608775naI1098366naI1837891naI1827507naI1381659naI1149100naI1821182naI150879305naI1152394naI656282naI496122naI514804naI533571naI1123881naI558671naI150024444naI755494naI151662814naI426455naI1985884naI471419naI579043naI2124227naI1490239naI456483naI620809naI1934159naI1543154naI150888013naI1808925naI713226naI614201naI1333292naI2057721naI1253144naI360189naI348638naI151636426naI1053108naI799261naI1324794naI1842691naI1022858naI672220naI448048naI1541355naI151683261naI617650naI2240294naI2229025naI451145naI2240115naI1806621naI2184379naI676192naI658865naI500572naI676211naI666345naI1818646naI922398naI147052169naI479322naI2244452naI1821377naI2343223naI1254061naI396443naI454991naI1861471naI1120789naI1188818naI610802naI2004597naI1853586naI1112393naI1299379naI1370283naI1813328naI1078572naI367661naI150133914naI447617naI1077576naI2054832naI1032223naI2102191naI2253138naI791040naI1045991naI948613naI615745naI2253132naI151683465naI151707518naI380665naI199861naI1966008naI2004054naI150136220naI483411naI197235naI150078347naI915311naI421109naI2279256naI150141084naI1196413naI1215552naI2369100naI1886360naI921051naI1097079naI150007781naI605442naI1500050naI2248761naI907720naI658246naI610859naI1813130naI1288854naI729814naI150970802naI151670783naI2016618naI1858932naI789864naI499203naI150985266naI1982418naI2106702naI637196naI959500naI1276176naI1500557naI151000765naI57969naI1364294naI1075231naI150073222naI2175437naI1845454naI207160naI955251naI1138334naI55705naI202172naI150089181naI150139956naI359096naI617227naI989813naI1867526naI621501naI2348033naI150162018naI905604naI2292926naI151662658naI152107844naI328440naI1821535naI368639naI1106025naI677161naI366731naI2226896naI531886naI1048248naI1049833naI2357301naI150150866naI150099687naI641518naI1174347naI1957091naI151679723naI2081725naI150068838naI257966naI289441naI1882009naI218485naI150024394naI478977naI1491754naI2087363naI2347699naI721296naI1316540naI1443575naI2218745naI323250naI150109759naI1400617naI2373502naI394691naI57686naI1414771naI150004574naI150102397naI1782540naI151712258naI2038578naI676424naI1993215naI612635naI1371027naI1434385naI1302470naI1542764naI151683300naI1050053naI1529323naI1148099naI717827naI1448545naI785482naI2173945naI570713naI1025750naI809994naI1205771naI784084naI712907naI1985263naI512910naI1173373naI841288naI273179naI384885naI600419naI1399431naI2226946naI2191208naI578678naI676515naI2238494naI1879484naI1402987naI2012441naI370658naI418394naI150004665naI2266752naI435620naI1860900naI413669naI2308756naI328663naI384868naI151683232naI1869389naI774433naI297542naI2046073naI858550naI759652naI554559naI1169765naI1470289naI2080256naI2090113naI497154naI996872naI150136155naI615166naI1842073naI151679514naI1293568naI328215naI1175437naI749192naI531905naI150882586naI151663533naI167537naI932042naI2063648naI145393219naI1466974naI150984297naI1388025naI532442naI362636naI884993naI1867993naI147549414naI150129972naI1867995naI545072naI216906naI2189184naI782273naI1305176naI1206399naI455922naI2233420naI1002643naI755143naI737823naI1332995naI150892727naI2167871naI421918naI1414027naI147651946naI1056793naI1513903naI1379788naI994975naI1152776naI2039877naI1557870naI151683126naI1317057naI439482naI35325naI1322148naI355783naI1838127naI1859018naI626642naI943778naI208830naI1366083naI1475461naI618566naI1820765naI1338247naI1303853naI935574naI491523naI2178325naI1111881naI1272411naI1325140naI1392442naI649651naI141579384naI1525734naI1403058naI1465204naI546832naI2254955naI1173177naI1076866naI2209499naI331891naI2188309naI1925129naI1305245naI150134117naI1785724naI1944078naI1946664naI399100naI670855naI1176588naI1942753naI1794204naI2227381naI712981naI2176077naI569305naI150980547naI2193913naI741997naI1856591naI638611naI1162751naI485564naI1516439naI1255933naI1857106naI1174011naI2265903naI151636628naI2195613naI2219470naI662611naI1372031naI1247859naI151652973naI783700naI1395397naI2238353naI622844naI602088naI2238450naI1260114naI1815571naI1564527naI809230naI2133829naI1270021naI359044naI1136642naI151634049naI330320naI1825270naI285120naI267630naI714188naI566835naI1966124naI601700naI1772909naI150972132naI998163naI805082naI898863naI1869570naI394814naI613504naI2041701naI2001764naI390516naI1316955naI150024601naI2043598naI258071naI1914875naI978998naI1452829naI719080naI1404611naI317955naI2089165naI223957naI339261naI519314naI1851371naI1858589naI666339naI1818802naI523669naI151663446naI1533500naI2179881naI2377997naI2298903naI1054228naI1041235naI327366naI1976317naI339278naI647800naI1906311naI813834naI225214naI2231023naI618254naI1465965naI1857893naI900666naI1805900naI2237497naI150851519naI1446111naI1857898naI898013naI1497468naI2199972naI1933647naI150141413naI1926405naI200768naI150966752naI1282212naI857924naI150980614naI2172785naI706749naI1053603naI177314naI225075naI1863568naI1771211naI150135633naI626607naI330335naI1832935naI279430naI1333337naI150966796naI2005155naI150888694naI2050029naI784911naI1122413naI150988760naI320563naI1394481naI606281naI57393naI1468519naI622743naI612483naI1148094naI1857768naI784648naI1868928naI944849naI1856543naI2204655naI428818naI456441naI748116naI1907084naI2212366naI1321855naI609517naI1329921naI1215803naI656643naI1547623naI357128naI2244308naI150165363naI1165552naI1050969naI2106914naI939705naI2049688naI1086084naI1023165naI1936128naI1904726naI965410naI151001009naI362675naI920031naI1421282naI54570naI258185naI698196naI150108068naI1946519naI150034097naI1322070naI2198525naI2198526naI2009080naI147507115naI54051naI1945546naI991733naI150893426naI636849naI1339899naI1856490naI2189372naI2090110naI2090111naI660961naI787181naI1949161naI1829284naI2056224naI496540naI773141naI1309476naI1889967naI485248naI150162602naI480865naI572349naI339116naI150136695naI151714099naI1829831naI150971024naI269169naI2235781naI2205327naI1245152naI238003naI198287naI855455naI623930naI1875202naI2330255naI917337naI732241naI1052018naI2349708naI283259naI443866naI1248911naI1876012naI1261266naI1863009naI1343452naI1874074naI1244761naI1938299naI649981naI613222naI1068186naI944756naI612629naI151683709naI1937674naI150034266naI274892naI611258naI1842080naI1426464naI541439naI638909naI1433685naI1317638naI1148930naI490118naI1419329naI1177524naI2257506naI2105111naI152108886naI1400714naI1174452naI1005571naI899310naI152104409naI47448naI196265naI2164303naI150966364naI1542749naI713875naI200854naI407144naI2164305naI1162156naI2324055naI2164309naI2295121naI151682794naI2351043naI203232naI390423naI190336naI1110604naI465501naI2026146naI1959486naI1354504naI1053950naI658906naI2011321naI1477804naI2197602naI2236115naI2298036naI878545naI2178983naI2086242naI1220802naI2356896naI152108081naI151707402naI230068naI1181241naI1380481naI150032823naI473258naI405156naI403191naI524847naI328788naI150971061naI1915322naI547008naI867583naI1868637naI1548366naI150140102naI996318naI634682naI210867naI1514239naI1418155naI1857605naI505297naI1962977naI1254704naI2073935naI2226523naI289075naI151667185naI2282406naI1106490naI1902768naI379118naI674623naI614070naI433844naI150033684naI1535551naI728631naI1138833naI1307109naI787553naI463688naI2015953naI150875420naI150873232naI150968591naI1395974naI712563naI2188039naI1813663naI1856766naI1913141naI1469190naI797620naI366960naI2109018naI1943583naI2102899naI1383634naI150970023naI1015789naI1184390naI897598naI59161naI952004naI1332569naI1158370naI2026064naI1413104naI150135346naI150982721naI732833naI1370455naI2037717naI2043718naI435226naI691659naI2354555naI2354554naI1966236naI1869965naI677799naI429777naI1139016naI977382naI332043naI475394naI1077561naI1805940naI1942808naI1978120naI1047990naI1190998naI150139931naI460184naI533970naI2234331naI329184naI150973110naI1381733naI147488840naI409588naI150973116naI2086570naI2191636naI1062603naI380129naI1946614naI229546naI2178332naI355922naI150089063naI1180129naI1404468naI150985390naI150149768naI602099naI1107362naI2217805naI1897139naI150112867naI1491212naI535619naI1148091naI463418naI1888342naI2010299naI1215656naI1315160naI2343038naI150140135naI1166089naI1280165naI564822naI1772911naI672616naI1443806naI1517057naI621559naI1516445naI376757naI622188naI794910naI341425naI1464453naI1152680naI1561181naI2189680naI1016760naI515629naI1996001naI1350972naI2308759naI2226916naI727937naI510318naI1002441naI390505naI1768488naI1398064naI2072954naI1189578naI563770naI2111156naI1457398naI150972260naI150975571naI2157094naI1517461naI1439153naI150989348naI1426962naI724826naI251140naI1390352naI1131843naI676173naI150989343naI250395naI569255naI1419796naI151634810naI150966846naI398792naI2246706naI1930974naI1821394naI485302naI547068naI785830naI449683naI1196943naI1330514naI1945109naI408240naI1053952naI633621naI2114207naI985237naI1051644naI1262175naI1191641naI1857737naI150062089naI500613naI859798naI366888naI1148095naI147505499naI2226562naI908486naI1560595naI2218291naI1219422naI666360naI699300naI1395870naI2366838naI481986naI2366835naI1047996naI150843500naI521293naI1470026naI1179539naI2164307naI2231689naI390223naI2010243naI1860783naI1882430naI2081663naI2183166na."},"resource":"search_area_restaurant_id:wwmthp","responsetime":31,"type":"redis"}

可以看到：

应用层协议：request 占了 52 字节（"bytes_in":52），response 占了 13507 字节（"bytes_out":13507）；
"responsetime" 仅为 31ms ；

抓包截图如下：

由此可知：

packetbeat 计算 "responsetime" 时是基于首个 TCP segment 计算的（在该抓包中，response 的最后一个 segment 和 request 之间的时间差为 54ms）；
"bytes_in" 和 "bytes_out" 的计算则是计算的全部 TCP segment 总和（应用协议数据长度）；

深入

确定产生上述日志的代码

func (redis *redisPlugin) newTransaction(requ,resp *redisMessage) common.MapStr {
    ...
	responseTime := int32(resp.ts.Sub(requ.ts).Nanoseconds() / 1e3)

	event := common.MapStr{
		"@timestamp":   common.Time(requ.ts),// requ.ts 即 request 包的时间戳
		"type":         "redis","status":       error,"responsetime": responseTime,// response 包的时间戳 - request 包的时间戳
		"redis":        returnValue,"method":       common.NetString(bytes.ToUpper(requ.method)),"resource":     requ.path,"query":        requ.message,"bytes_in":     uint64(requ.size),"bytes_out":    uint64(resp.size),"src":          src,"dst":          dst,}
    ...
}

request 和 response 的关联方式

// 将 request 和 response 进行关联
func (redis *redisPlugin) correlate(conn *redisConnectionData) {
    ...
    // merge requests with responses into transactions
	for !conn.responses.empty() && !conn.requests.empty() {
		requ := conn.requests.pop()
		resp := conn.responses.pop()

		if redis.results != nil {
			// 将匹配的 request 和 response 封装成 transaction event
			event := redis.newTransaction(requ,resp)
			// 将 transaction event 发布出去（比如写出到 stdout 或 file 等）
			redis.results.PublishTransaction(event)
		}
	} 
    ...
}

这里其实存在一个问题：上述实现中由于采用的是 FIFO 原则，因此必须确保 request 和 response 是正确匹配的，但在 redis 协议的 master 和 slave 之间进行命令同步相关协议通信时，不满足该要求，会导致匹配错误产生；

将 redis 协议中的 request 和 response 分类保存和匹配

func (redis *redisPlugin) handleRedis(
	conn *redisConnectionData,m *redisMessage,tcptuple *common.TCPTuple,dir uint8,) {
    ...

	if m.isRequest {
		conn.requests.append(m) // wait for response
	} else {
		conn.responses.append(m)
		redis.correlate(conn)
	}
}

设置 request 或 response 时间戳的地方

func (redis *redisPlugin) doParse(
	conn *redisConnectionData,pkt *protos.Packet,) *redisConnectionData {

	// 基于当前 tcp 流的数据流向进行选择处理
	st := conn.streams[dir]
	if st == nil {
		// 新建 stream 时，为当前 redis 消息指定时间戳，即新 request 或 response 
		st = newStream(pkt.Ts,tcptuple)   -- 1
		conn.streams[dir] = st
		if isDebug {
			debugf("new stream: %p (dir=%v,len=%v)",st,dir,len(pkt.Payload))
		}
	}
	...
	for st.Buf.Len() > 0 {
		if st.parser.message == nil {
			// 为新的一条 redis 消息指定时间戳，即新 request 或 response
			st.parser.message = newMessage(pkt.Ts)  -- 2
		}
   ...
   redis.handleRedis(conn,msg,tcptuple,dir)
   ...
}

可以看到时间戳的指定是在收到 redis 协议 request 或 response 的第一个 pkt 时进行的；

TCP 协议包到应用层协议包的处理

// 将 tcp 协议层面的数据包添加到
func (stream *TCPStream) addPacket(pkt *protos.Packet,tcphdr *layers.TCP) {
    ...
	if len(pkt.Payload) > 0 {
		// 调用各协议模块定义的 Parse 函数
		conn.data = mod.Parse(pkt,&conn.tcptuple,stream.dir,conn.data)
	}
    ...
}

将 pkt 分配到不同的 TCP stream

func (tcp *TCP) Process(id *flows.FlowID,tcphdr *layers.TCP,pkt *protos.Packet) {
    ...
	// 基于 pkt 确定 TCP stream
	stream,created := tcp.getStream(pkt)
	...
	stream.addPacket(pkt,tcphdr)
}

针对基于 gopacket 从底层收上来的 packet 进行协议的逐层解析

func (d *Decoder) OnPacket(data []byte,ci *gopacket.CaptureInfo) {
    ...
	// Ethernet 层给出的类型
	currentType := d.linkLayerType
	
	// NOTE: 这里就是为每个 packet 设置时间戳的位置
	//       可以看到时间戳实际上是来自 gopacket 中给的值
	packet := protos.Packet{Ts: ci.Timestamp}
    ...
    for len(data) > 0 {
        ...
		// 根据 packet 所属的 layerType 触发相应的回调函数(ICMP/UDP/TCP)
		processed,err = d.process(&packet,currentType)
        ...
    }
}
...
// 根据 packet 所属的 layerType 触发相应的回调函数(ICMP/UDP/TCP)
func (d *Decoder) process(
	packet *protos.Packet,layerType gopacket.LayerType,) (bool,error) {
    ...
	case layers.LayerTypeTCP:
		debugf("TCP packet")
		d.onTCP(packet)
		return true,nil
	}
    ...   
}
...
func (d *Decoder) onTCP(packet *protos.Packet) {
    ...
	d.tcpProc.Process(id,&d.tcp,packet)
}

sniffer 根据配置 DataSource 读取数据包

func (sniffer *SnifferSetup) Run() error {
    ...
	for sniffer.isAlive {
	    ...
		// 从指定数据源（live interface 或者 pcap 文件）中获取下一个存在的 packet 
		//  data:  The bytes of an individual packet.
	    //  ci:  Metadata about the capture
		data,ci,err := sniffer.DataSource.ReadPacketData()
		...
		sniffer.worker.OnPacket(data,&ci)
	}
    ...
}

由上下文可知，时间戳是从 ci 中得到的，因此 DataSource 使用哪种是关键；

数据源 DataSource 的选择

func (sniffer *SnifferSetup) setFromConfig(config *config.InterfacesConfig) error {
    ...
	switch sniffer.config.Type {
	case "pcap":
	    ...
	    sniffer.DataSource = gopacket.PacketDataSource(sniffer.pcapHandle)
	case "af_packet":
	    ...
	    sniffer.DataSource = gopacket.PacketDataSource(sniffer.afpacketHandle)
	case "pfring","pf_ring":
	    ...
	    sniffer.DataSource = gopacket.PacketDataSource(sniffer.pfringHandle)
	default:
		return fmt.Errorf("Unknown sniffer type: %s",sniffer.config.Type)
	}
	...
}

对应测试中的实际情况而言，此处只考虑 pcap 对应的情况；

ci 中时间戳的设置

在 pcap.go 中有

func (p *Handle) ReadPacketData() (data []byte,ci gopacket.CaptureInfo,err error) {
    ...
	err = p.getNextBufPtrLocked(&ci)
    ...
}
...
func (p *Handle) getNextBufPtrLocked(ci *gopacket.CaptureInfo) error {
    ...
    // 设置 ci 中时间戳的位置
	ci.Timestamp = time.Unix(int64(p.pkthdr.ts.tv_sec),int64(p.pkthdr.ts.tv_usec)*1000) // convert micros to nanos
	ci.CaptureLength = int(p.pkthdr.caplen)
	ci.Length = int(p.pkthdr.len)
    ...
}

可以看到，时间戳是从 p.pkthdr.ts.tv_sec 和 p.pkthdr.ts.tv_usec 两个值得到的，而 pkthdr 的定义如下：

type Handle struct {
	// cptr is the handle for the actual pcap C object.
	cptr         *C.pcap_t
	blockForever bool
	device       string
	mu           sync.Mutex
	// Since pointers to these objects are passed into a C function,if
	// they're declared locally then the Go compiler thinks they may have
	// escaped into C-land,so it allocates them on the heap.  This causes a
	// huge memory hit,so to handle that we store them here instead.
	pkthdr  *C.struct_pcap_pkthdr
	buf_ptr *C.u_char
}

由此可知，要想知道 pkthdr.ts 的值是如何得到的，则需要研究 libpcap 代码了；

展开

在《Timestamps》中，有如下说明：

Wireshark just gets its timestamp from libpcap/WinPcap,and libpcap/WinPcap gets it from the packet capture mechanism it uses; Wireshark itself doesn't generate the timestamp so there's nothing Wireshark can do about it.

How the timestamp works is OS dependent. In some UNIXes that code is in the network drivers; it's higher up in the networking code path in other UNIXes. In Windows,with WinPcap,timestamping is done by the WinPcap driver.

Another issue might be if the OS delivers multiple packets per interrupt. This can be because the OS does "polling" or otherwise arranges that one interrupt be delivered for a batch of packets to reduce interrupt-handling overhead. If that's the case,the timestamps might be the same for multiple packets,at least to the resolution of the timestamping routine.

Note also that the timestamp on a packet isn't a high-accuracy measurement of when the first bit or the last bit of the packet arrived at the network adapter; there's a delay between the arrival of that last bit and the interrupt for the packet and a delay between the start of interrupt handling and the point in the code path where the timestamp is attached to the packet.

Resolution

The timestamp has the resolution of whatever clock is being used. The clock might not be the "PC clock" because it might not be running on a "PC",either in the sense of machines sold as "personal computers" or in the sense of "IBM-compatible personal computer". Some of those machines might have better high-resolution timers than IBM-compatible PCs do - at least some OSes on more modern IBM-compatible PC's use the RDTSC instruction,if present on the processor,to get higher-precision timestamps.

There's precision and accuracy; a clock with picosecond resolution,set to a time that's 1 1/2 hours off,is very precise and very inaccurate.

Discussion

That's the usual discussion about this might be / this could be / this will be / and so on.

Please put some hard facts here ...

Just simply add measurement values (and the hard facts on the environment) on a specific platform so others can participate ...

在《gettimeofday() should never be used to measure time》中，有如下说明：

The pcap_pkthdr struct (the “received packet” struct) contains a struct timeval ts that ruins our ability to measure the time it takes for the reply you get for some query you sent. They tell me the kernel supplies the timestamp,so it’s not really libpcaps fault.

Calling clock_gettime() when libpcap gives you a packet has turned out to be useless,as the time difference between packet reception and the delivery to your program is too long and unstable. You’re stuck with this wall-clock time until you fix all the kernels in the world and break binary compatibility with old libpcap programs.

在《how to understand the capture time!》中，有如下精彩讨论：

[Ask #1]

Hi,Dear All,

The webpages about pcap says that the "pcap_pkthdr" structure contains the information about when the packet was sniffed,that is:
struct pcap_pkthdr{ 
    struct timeval ts;
    bpf_u_int32 caplen;
    bpf_u_int32 len;
}
I wonder whether the "ts" is just the time when the pcap captured the packet? Whether Ethereal use this data for the time when a packet was captured?

Ethereal display the captured packets like:
Frame 1 
Arrival time: Jun 13,2002 12:00:00.1234546789 
?? ...
How Ethereal gets this arrival time? from the pcap_pkthdr mentioned upper? the datum "123456789" come directly from the "tv_usec" part in the timeval strcuture?

[Answer #1]

What "ts" means depends on the operating system on which you're capturing packets.

On most operating systems,it's the time at which the driver for the network interface gave the packet to the OS's packet capture mechanism; On some OSes where the operating system doesn't itself put a timestamp on the packet,it's the time at which the libpcap library read the packet from the OS kernel.

I.e.,the time isn't necessarily the time when the packet arrived on the machine running tcpdump/Ethereal/whatever sniffer program you're using - it may be a later time (although it probably won't be much later).

The answers to all these questions are "YES":

Whether Ethereal use this data for the time when a packet was captured?

How Ethereal gets this arrival time? from the pcap_pkthdr mentioned upper?

the datum "123456789" come directly from the "tv_usec" part in the timeval strcuture?

Note that not all OSes necessarily provide high-precision timestamps; they might,for example,provide timestamps with 1 millisecond or 10 millisecond resolution.

[Ask #2]

However,I still don't understand the capture time Ethereal display. for example,when I capture the icmp packet produced by "ping host B" on host A,it shows the same capture time of echo request and echo reply,as the following:
1 0.000000 A B ICMP Echo(ping) request 
Arrival Time: Jun 14,2002 12:00:00.123456789 
... 
2 0.000000 B A ICMP Echo(ping) reply 
Arrival Time: Jun 14,2002 12:00:00.123456789 
...
I wonder why the set of icmp packets arrive at the same time? since A ping B,and B returns a echo reply,it shouldn't produce at the same time!

More ever,I captured the "A ping B" echo request packet on host B,and I want to compute the transmit time for the packet.(A and B have been synchronized by NTP time server)

But transmit = "the arrvial time on host B" subtract "the time of the echo request produced on host A"

the transmit seems much different from the "round-trip time"/2 displayed by "ping",I mean,it seems they are not in the same quantity scale. So I feel confused. Would you like to give me some suggestion?

[Answer #2]

You're assuming they did arrive at the same time; that may not be the case.

Perhaps the OS on which you're running the program on which you're capturing packets doesn't timestamp the packets with a sufficiently high-resolution time stamp,and doesn't try to give packets unique time stamps,either.

If,the reply was received .1 milliseconds after the request was sent,but the timer the OS uses to time-stamp the packets has only a 1-millisecond resolution,the two packets might be given the same time stamp even though the request was sent at a different time from when the reply arrived.

There's nothing Ethereal can do about that; it just displays the times libpcap gave it (or gave whatever program wrote the capture file).

There's probably not much libpcap can do about that,either; it just gets the times that the OS provides.

在《[tcpdump-workers] Monotonic clock timestamp on packets 》中，提到

[Ask #1]

Has anyone looked into timestamping the captured packets using clock_gettime(CLOCK_MONOTONIC,...)?

I'm thinking adding a struct timespec to struct pcap_pkthdr and filling that in addition to the struct timeval.

For a request-reply situation a monotonic clock is much more reliable than gettimeofday().

[Answer #1]

pcap_pkthdr is in a file. You cannot add ANYTHING to it without breaking compatibility; you'd have to introduce a new magic number.

BTW,note that if you call clock_gettime(),there is NO guarantee that the time it returns has anything to do with the time the packe arrived; it tells you the time when it's called,not the time when the packet arrived.

The only platforms on which libpcap uses gettimeofday() are:

DLPI platforms - where the DLPI module doesn't supply the time stamp (e.g.,HP-UX);

DOS;

Septel devices;

USB capturing on Linux if you're not using the memory-mapped interface.

On all other platforms - i.e.,on most of the platforms where libpcap is used - the time stamps are supplied to userland by the kernel,so if you want to use a different timer,you'll have to modify the kernel. (*BSD,Mac OS X,Linux,Solaris,etc.)

[Ask #2]

Exactly. That's why I asked if anyone has taken a look at it. Because calling it from the application at pcap_dispatch time would be useless. Just like calling it from libpcap an arbitrary time too late would be useless.

So if the underlying systems don't provide a monotonic clock for packet arrival time then that's that.

结论：

在有些系统上，这个时间戳对应的是网卡驱动将数据交付 OS 内核的时间；
在另外一些系统上，这个时间戳对应的是负责 sniffer 的东东（例如 libpcap）从 OS 内核读取数据包的时间；
应该认为这个时间戳并非准确的，尤其是在系统压力比较大的情况下；

其他

[tcpdump-workers] Receive timestamp
[tcpdump-workers] libpcap based timestamp in linux
[tcpdump-workers] libpcap and select problem
CINBAD investigation of different packet filters
Libpcap tutorial
mail-archive
gt.net

（编辑：李大同）

【声明】本站内容均来自网络，其相关言论仅代表作者个人观点，不代表本站立场。若无意侵犯到您的权利，请及时与联系站长删除相关内容!