Shell脚本处理JSON数据工具jq

发布时间：2020-12-15 23:13:48 所属栏目：安全来源：网络整理

导读：shell脚本如何方便地处理JSON格式的数据呢，这里介绍一个工具：jq 使用参数介绍：https://stedolan.github.io/jq/manual/ ? 官方教程简单翻译如下。 1、获取JSON数据我们以github上jq项目最新5条评论的JSON数据为例。获取数据如下: curl ‘https://api.git

shell脚本如何方便地处理JSON格式的数据呢，这里介绍一个工具：jq

使用参数介绍：https://stedolan.github.io/jq/manual/

官方教程简单翻译如下。

1、获取JSON数据

我们以github上jq项目最新5条评论的JSON数据为例。获取数据如下:

curl ‘https://api.github.com/repos/stedolan/jq/commits?per_page=5‘

结果如下：

[
  {
    "sha": "d25341478381063d1c76e81b3a52e0592a7c997f","commit": {
      "author": {
        "name": "Stephen Dolan","email": "[email?protected]","date": "2013-06-22T16:30:59Z"
      },"committer": {
        "name": "Stephen Dolan","email": "[email?protected]","message": "Merge pull request #162 from stedolan/utf8-fixesnnUtf8 fixes. Closes #161","tree": {
        "sha": "6ab697a8dfb5a96e124666bf6d6213822599fb40","url": "https://api.github.com/repos/stedolan/jq/git/trees/6ab697a8dfb5a96e124666bf6d6213822599fb40"
      },"url": "https://api.github.com/repos/stedolan/jq/git/commits/d25341478381063d1c76e81b3a52e0592a7c997f","comment_count": 0
    },"url": "https://api.github.com/repos/stedolan/jq/commits/d25341478381063d1c76e81b3a52e0592a7c997f","html_url": "https://github.com/stedolan/jq/commit/d25341478381063d1c76e81b3a52e0592a7c997f","comments_url": "https://api.github.com/repos/stedolan/jq/commits/d25341478381063d1c76e81b3a52e0592a7c997f/comments","author": {
      "login": "stedolan",...

2、高亮并按属性排序显示

用 jq ‘.‘即可：

curl ‘https://api.github.com/repos/stedolan/jq/commits?per_page=5‘ | jq ‘.‘

结果：

[
  {
    "parents": [
      {
        "html_url": "https://github.com/stedolan/jq/commit/54b9c9bdb225af5d886466d72f47eafc51acb4f7","url": "https://api.github.com/repos/stedolan/jq/commits/54b9c9bdb225af5d886466d72f47eafc51acb4f7","sha": "54b9c9bdb225af5d886466d72f47eafc51acb4f7"
      },{
        "html_url": "https://github.com/stedolan/jq/commit/8b1b503609c161fea4b003a7179b3fbb2dd4345a","url": "https://api.github.com/repos/stedolan/jq/commits/8b1b503609c161fea4b003a7179b3fbb2dd4345a","sha": "8b1b503609c161fea4b003a7179b3fbb2dd4345a"
      }
    ],"committer": {
      "type": "User","received_events_url": "https://api.github.com/users/stedolan/received_events","events_url": "https://api.github.com/users/stedolan/events{/privacy}","repos_url": "https://api.github.com/users/stedolan/repos","organizations_url": "https://api.github.com/users/stedolan/orgs",...

这里的评论内容比较多，我们现在想拿第一个评论。

3、获取数组某一项

curl ‘https://api.github.com/repos/stedolan/jq/commits?per_page=5‘ | jq ‘.[0]‘

结果：

? View Code

可以看到，我们已经拿到了一条完整的评论内容。但我们真正关心的只是评论内容和用户名，下面来获取这两项内容。

4、自定义格式输出某一项

jq ‘.[0] | {message: .commit.message,name: .commit.committer.name}‘

结果：

{
  "name": "Stephen Dolan","message": "Merge pull request #162 from stedolan/utf8-fixesnnUtf8 fixes. Closes #161"
}

我们可以看到，已经拿到了想要的内容，并且已经按我们自己定义的格式显示了。

这里?|?后面的内容是以前面的内容为输入的，.commit?中的?.?就是指?.[0]?中的内容。

5、自定义格式输出多项

jq ‘.[] | {message: .commit.message,"message": "Merge pull request #162 from stedolan/utf8-fixesnnUtf8 fixes. Closes #161"
}
{
  "name": "Stephen Dolan","message": "Reject all overlong UTF8 sequences."
}
{
  "name": "Stephen Dolan","message": "Fix various UTF8 parsing bugs.nnIn particular,parse bad UTF8 by replacing the broken bits with U+FFFDnand resychronise correctly after broken sequences."
}
{
  "name": "Stephen Dolan","message": "Fix example in manual for `floor`. See #155."
}
{
  "name": "Nicolas Williams","message": "Document floor"
}

这里 .[] 获取的是数组中的所有项。

我们看到，结果是一个个独立的JSON对象，如何把结果组合成一个数组呢？

6、以数组形式自定义输出多项

jq ‘[.[] | {message: .commit.message,name: .commit.committer.name}]‘

结果：

[
  {
    "name": "Stephen Dolan","message": "Merge pull request #162 from stedolan/utf8-fixesnnUtf8 fixes. Closes #161"
  },{
    "name": "Stephen Dolan","message": "Reject all overlong UTF8 sequences."
  },parse bad UTF8 by replacing the broken bits with U+FFFDnand resychronise correctly after broken sequences."
  },"message": "Fix example in manual for `floor`. See #155."
  },{
    "name": "Nicolas Williams","message": "Document floor"
  }
]

我们可以看到，只要在上一步的命令中内容的两端加个中括号即可。

最后，我们如果想获取每个评论的引用评论的url（在parents节点中，有一个或多个）呢？

7、获取其他内容

jq ‘[.[] | {message: .commit.message,name: .commit.committer.name,parents: [.parents[].html_url]}]‘

结果：

[
  {
    "parents": [
      "https://github.com/stedolan/jq/commit/54b9c9bdb225af5d886466d72f47eafc51acb4f7","https://github.com/stedolan/jq/commit/8b1b503609c161fea4b003a7179b3fbb2dd4345a"
    ],"name": "Stephen Dolan",{
    "parents": [
      "https://github.com/stedolan/jq/commit/ff48bd6ec538b01d1057be8e93b94eef6914e9ef"
    ],{
    "parents": [
      "https://github.com/stedolan/jq/commit/54b9c9bdb225af5d886466d72f47eafc51acb4f7"
    ],{
    "parents": [
      "https://github.com/stedolan/jq/commit/3dcdc582ea993afea3f5503a78a77675967ecdfa"
    ],{
    "parents": [
      "https://github.com/stedolan/jq/commit/7c4171d414f647ab08bcd20c76a4d8ed68d9c602"
    ],"name": "Nicolas Williams","message": "Document floor"
  }
]

这里用?.parents[].html_url?获取当前项的?parents?节点中的所有项的?html_url?属性的内容，然后两边加个中括号组装成数组输出。

怎么样，经过这个例子可以看出，用jq处理JSON数据还是很方便强大的吧。

8、注意点

jq能处理的需要是严格的JSON格式数据，JSON对象和JSON字符串是不行的，如下面的两种格式数据jq是不能处理的：

json对象：

{
    a: 1,b: {
        c: "abc"
    }
}

json字符串：

‘{"a":1,"b":{"c":"abc"}}‘

正确的JSON格式：

{
    "a": 1,"b": {
        "c": "abc"
    }
}

关于什么是JSON格式数据，请参考：http://www.json.org/

（编辑：李大同）

【声明】本站内容均来自网络，其相关言论仅代表作者个人观点，不代表本站立场。若无意侵犯到您的权利，请及时与联系站长删除相关内容!