shell编写streaming

发布时间：2020-12-15 16:43:50 所属栏目：安全来源：网络整理

导读：##mapper.sh#! /bin/bashwhile read LINE; do for word in $LINE do echo "$word 1" donedone##reducer.sh#! /bin/bashcount=0started=0word=""while read LINE;do newword=`echo $LINE | cut -d ' ' -f 1` if [ "$word" != "$newword" ];then [ $started -n

##mapper.sh
#! /bin/bash
while read LINE; do
  for word in $LINE
  do
    echo "$word 1"
  done
done



##reducer.sh
#! /bin/bash
count=0
started=0
word=""
while read LINE;do
  newword=`echo $LINE | cut -d ' '  -f 1`
  if [ "$word" != "$newword" ];then
    [ $started -ne 0 ] && echo -e "$word t $count"
    word=$newword
    count=1
    started=1
  else
    count=$(( $count + 1 ))
  fi
done
echo -e "$word t $count"

（编辑：李大同）

【声明】本站内容均来自网络，其相关言论仅代表作者个人观点，不代表本站立场。若无意侵犯到您的权利，请及时与联系站长删除相关内容!