oracle – 如何在运行Sqoop导入和导出时找到最佳映射器数量？

运行Sqoop命令时.例如这样：

./sqoop import                               
    --fs <name node>                         
    --jt <job tracker>                       
    --connect <JDBC string>                  
    --username <user> --password <password>  
    --table <table> --split-by <cool column> 
    --target-dir <where>                     
    --verbose --m 2

我们可以指定-m – 我们希望Sqoop运行多少并行任务(也可能同时访问数据库).
相同选项适用于./sqoop export< ...>

是否有一些启发式(可能基于数据大小)有助于猜测什么是最佳使用任务数？

谢谢！

The optimal number of mappers depends on many variables: you need to take into account your database type,the hardware that is used for your database server,and the impact to other requests that your database needs to serve. There is no optimal number of mappers that works for all scenarios. Instead,you’re encouraged to experiment to find the optimal degree of parallelism for your environment and use case. It’s a good idea to start with a small number of mappers,slowly ramping up,rather than to start with a large number of mappers,working your way down.

（编辑：李大同）

【声明】本站内容均来自网络，其相关言论仅代表作者个人观点，不代表本站立场。若无意侵犯到您的权利，请及时与联系站长删除相关内容!