php – 如何处理大型帖子请求
不幸的是,我无法向您展示代码,但我可以让您了解它的外观,它的作用以及我遇到的问题……
<?php include(db.php); include(tools.php); $c = new GetDB(); // Connection to DB $t = new Tools(); // Classes to clean,prevent XSS and others if(isset($_POST['var'])){ $nv = json_decode($_POST['var']) foreach($nv as $k) { $id = $t->clean($k->id); // ... goes on for about 10 keys // this might seems redundant or insufficient $id = $c->real_escape_string($id); // ... goes on for the rest of keys... $q = $c->query("SELECT * FROM table WHERE id = '$id'"); $r = $q->fetch_row(); if ($r[1] > 0) { // Item exist in DB then just UPDATE $q1 = $c->query(UPDATE TABLE1); $q4 = $c->query(UPDATE TABLE2); if ($x == 1) { $q2 = $c->query(SELECT); $rq = $q2->fetch_row(); if ($rq[0] > 0) { // Item already in table just update $q3 = $c->query(UPDATE TABLE3); } else { // Item not in table then INSERT $q3 = $c->query(INSERT TABLE3); } } } else { // Item not in DB then Insert $q1 = $c->query(INSERT TABLE1); $q4 = $c->query(INSERT TABLE2); $q3 = $c->query(INSERT TABLE4); if($x == 1) { $q5 = $c->query(INSERT TABLE3); } } } } 正如你所看到的那样是一个非常基本的INSERT,UPDATE表脚本,所以在我们发布到完全生产之前,我们做了一些测试,看看脚本是否正常工作,并且“结果”非常好…… array ( [0] => array ( [id] => 1,[val2] => 1,[val3] => 1,[val4] => 1,[val5] => 1,[val6] => 1,[val7] => 1,[val8] => 1,[val9] => 1,[val10] => 1 ),[1] => array ( [id] => 2,[val2] => 2,[val3] => 2,[val4] => 2,[val5] => 2,[val6] => 2,[val7] => 2,[val8] => 2,[val9] => 2,[val10] => 2 ),//... about 10 to 20K depend on the day and time ) 但在json …任何方式,发送此信息不是问题,就像我说它可能需要大约3到5分钟,问题是执行接收数据的工作的代码并执行查询…在正常共享托管我们得到一个503错误,通过调试它结果是一个超时,所以对于我们的VPS我们可以将max_execution_time增加到我们需要的任何东西,处理10K我们的VPS需要大约1小时,但在共享托管,我们不能使用max_execution_time …所以我问其他开发人员发送信息,而不是一次发送10K发送一批1K,让它休息一秒然后发送另一批… …等等……到目前为止我还没有得到任何答案……所以我想在我的结束时做“停顿”,比如,在过程1K项目等待一秒然后继续但是我没有看到它像批量接收数据一样高效…你会如何解决这个问题? 解决方法
对不起,我没有足够的声誉可以在任何地方发表评论,但是,我必须在答案中写下这个.我会推荐zedfoxus上面的批处理方法.此外,我强烈建议找出一种更快地处理这些查询的方法.请记住,每一个PHP函数调用等都会乘以每一行数据.以下是您可以获得更好性能的几种方法:
>使用准备好的陈述.这将允许MySQL缓存每个连续查询的内存操作.这非常重要. 一个.考虑创建一个临时的预编译数组(取决于所涉及的内存使用情况),该数组存储唯一的数据行.我也会考虑为辅助TABLE3做同样的事情.这将消除不必要的“更新”查询,并使b部分成为可能. 湾考虑单个查询,从数组中的数据库中选择每个id.这将是使用UPDATE查询的项目列表.更新每个行,然后将其从临时数组中删除.然后,您可以创建单个多行插入语句(当然是准备好的),它一次完成所有插入操作. 我希望这有帮助,如果其他人有一些建议,只需在评论中发布它们,我会尝试将它们包括在内. 以下是原始代码,只提供一些更改建议: <?php /* You can make sure that the connection type is persistent and * I personally prefer using the PDO driver. */ include(db.php); /* Definitely think twice about each tool that is included. * Only include what you need to evaluate the submitted data. */ include(tools.php); $c = new GetDB(); // Connection to DB /* Take a look at optimizing the code in the Tools class. * Avoid any and all kinds of loops–this code is going to be used in * a loop and could easily turn into O(n^2) performance drain. * Minimize the amount of string manipulation requests. * Optimize regular expressions. */ $t = new Tools(); // Classes to clean,prevent XSS and others if(isset($_POST['var'])){ // !empty() catches more cases than isset() $nv = json_decode($_POST['var']) /* LOOP LOGIC * Definitely test my hypothesis yourself,but this is similar * to what I would try first. */ //Row in database query $inTableSQL = "SELECT id FROM TABLE1 WHERE id IN("; //keep adding to it foreach ($nv as $k) { /* I would personally use specific methods per data type. * Here,I might use a type cast,plus valid int range check. */ $id = $t->cleanId($k->id); //I would include a type cast: (int) // Similarly for other values //etc. // Then save validated data to the array(s) $data[$id] = array($values...); /* Now would also be a good time to add the id to the SELECT * statement */ $inTableSQL .= "$id,"; } $inTableSQL .= ");"; // Execute query here // Then step through the query ids returned,perform UPDATEs,// remove the array element once UPDATE is done (use prepared statements) foreach (..... /* Then,insert the remaining rows all at once... * You'll have to step through the remaining array elements to * prepare the statement. */ foreach(..... } //end initial POST data if /* Everything below here becomes irrelevant */ foreach($nv as $k) { $id = $t->clean($k->id); // ... goes on for about 10 keys // this might seems redundant or insufficient $id = $c->real_escape_string($id); // ... goes on for the rest of keys... $q = $c->query("SELECT * FROM table WHERE id = '$id'"); $r = $q->fetch_row(); if ($r[1] > 0) { // Item exist in DB then just UPDATE $q1 = $c->query(UPDATE TABLE1); $q4 = $c->query(UPDATE TABLE2); if ($x == 1) { $q2 = $c->query(SELECT); $rq = $q2->fetch_row(); if ($rq[0] > 0) { // Item already in table just update $q3 = $c->query(UPDATE TABLE3); } else { // Item not in table then INSERT $q3 = $c->query(INSERT TABLE3); } } } else { // Item not in DB then Insert $q1 = $c->query(INSERT TABLE1); $q4 = $c->query(INSERT TABLE2); $q3 = $c->query(INSERT TABLE4); if($x == 1) { $q5 = $c->query(INSERT TABLE3); } } } } (编辑:李大同) 【声明】本站内容均来自网络,其相关言论仅代表作者个人观点,不代表本站立场。若无意侵犯到您的权利,请及时与联系站长删除相关内容! |