PHP致命错误：第62行上的NormalizerFormatter.php上允许的内存大

发布时间：2020-12-13 22:10:53 所属栏目：PHP教程来源：网络整理

导读：我试图通过doctrine2在 mysql中插入大约20000条记录 foreach ($getIds as $id){// 20000 Ids $twNode = new TwNode(); $twNode-setTwId($id); $twNode-addAcFriend($this-twAccount); $this-twAccount-addFollower($twNode); $this-em-persist($twNode); $th

我试图通过doctrine2在 mysql中插入大约20000条记录

foreach ($getIds as $id){// 20000 Ids
            $twNode = new TwNode();
            $twNode->setTwId($id);
            $twNode->addAcFriend($this->twAccount);
            $this->twAccount->addFollower($twNode);
            $this->em->persist($twNode);
            $this->em->persist($this->twAccount);
            $this->em->flush();
            $this->output->write('W');
            $followersWrite++;
        }
    }

但是这个错误很愉快,我怎样才能避免这个允许的内存错误？

PHP Fatal error:  Allowed memory size of 268435456 bytes exhausted (tried to allocate 72 bytes) in /Users/whitebear/httproot/myapp/vendor/monolog/monolog/src/Monolog/Formatter/NormalizerFormatter.php on line 62
PHP Stack trace:
PHP   1. {main}() /Users/whitebear/httproot/myapp/app/console:0
PHP   2. SymfonyComponentConsoleApplication->run() /Users/whitebear/httproot/myapp/app/console:22
PHP   3. SymfonyBundleFrameworkBundleConsoleApplication->doRun() /Users/whitebear/httproot/myapp/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:106
PHP   4. SymfonyComponentConsoleApplication->doRun() /Users/whitebear/httproot/myapp/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php:83
PHP   5. SymfonyComponentConsoleCommandCommand->run() /Users/whitebear/httproot/myapp/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:200
PHP   6. AcmeTopBundleCommandtwCrawlerCommand->execute() /Users/whitebear/httproot/myapp/vendor/symfony/symfony/src/Symfony/Component/Console/Command/Command.php:242
PHP   7. AcmeTopBundleCommandtwCrawlerCommand->initialRun() /Users/whitebear/httproot/myapp/src/Acme/TopBundle/Command/twCrawlerCommand.php:430
PHP   8. EntityManager52ee6dcbdb2bb_546a8d27f194334ee012bfe64f629947b07e4919__CG__DoctrineORMEntityManager->flush() /Users/whitebear/httproot/myapp/src/Acme/TopBundle/Command/twCrawlerCommand.php:284
PHP   9. DoctrineORMEntityManager->flush() /Users/whitebear/httproot/myapp/app/cache/dev/jms_diextra/doctrine/EntityManager_52ee6dcbdb2bb.php:402
PHP  10. DoctrineORMUnitOfWork->commit() /Users/whitebear/httproot/myapp/vendor/doctrine/orm/lib/Doctrine/ORM/EntityManager.php:389
PHP  11. DoctrineORMUnitOfWork->executeInserts() /Users/whitebear/httproot/myapp/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php:335
PHP  12. DoctrineORMPersistersBasicEntityPersister->executeInserts() /Users/whitebear/httproot/myapp/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php:952
PHP  13. DoctrineDBALStatement->execute() /Users/whitebear/httproot/myapp/vendor/doctrine/orm/lib/Doctrine/ORM/Persisters/BasicEntityPersister.php:279
PHP  14. DoctrineDBALLoggingLoggerChain->startQuery() /Users/whitebear/httproot/myapp/vendor/doctrine/dbal/lib/Doctrine/DBAL/Statement.php:161
PHP  15. SymfonyBridgeDoctrineLoggerDbalLogger->startQuery() /Users/whitebear/httproot/myapp/vendor/doctrine/dbal/lib/Doctrine/DBAL/Logging/LoggerChain.php:54
PHP  16. SymfonyBridgeDoctrineLoggerDbalLogger->log() /Users/whitebear/httproot/myapp/vendor/symfony/symfony/src/Symfony/Bridge/Doctrine/Logger/DbalLogger.php:80
PHP  17. MonologLogger->debug() /Users/whitebear/httproot/myapp/vendor/symfony/symfony/src/Symfony/Bridge/Doctrine/Logger/DbalLogger.php:102
PHP  18. MonologLogger->addRecord() /Users/whitebear/httproot/myapp/vendor/monolog/monolog/src/Monolog/Logger.php:423
PHP  19. MonologHandlerAbstractProcessingHandler->handle() /Users/whitebear/httproot/myapp/vendor/monolog/monolog/src/Monolog/Logger.php:244
PHP  20. MonologFormatterLineFormatter->format() /Users/whitebear/httproot/myapp/vendor/monolog/monolog/src/Monolog/Handler/AbstractProcessingHandler.php:35
PHP  21. MonologFormatterNormalizerFormatter->format() /Users/whitebear/httproot/myapp/vendor/monolog/monolog/src/Monolog/Formatter/LineFormatter.php:43
PHP  22. MonologFormatterLineFormatter->normalize() /Users/whitebear/httproot/myapp/vendor/monolog/monolog/src/Monolog/Formatter/NormalizerFormatter.php:40
PHP  23. MonologFormatterNormalizerFormatter->normalize() /Users/whitebear/httproot/myapp/vendor/monolog/monolog/src/Monolog/Formatter/LineFormatter.php:88
PHP  24. MonologFormatterLineFormatter->normalize() /Users/whitebear/httproot/myapp/vendor/monolog/monolog/src/Monolog/Formatter/NormalizerFormatter.php:70
PHP  25. MonologFormatterNormalizerFormatter->normalize() /Users/whitebear/httproot/myapp/vendor/monolog/monolog/src/Monolog/Formatter/LineFormatter.php:88

Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 72 bytes) in /Users/whitebear/httproot/myapp/vendor/monolog/monolog/src/Monolog/Formatter/NormalizerFormatter.php on line 62

Call Stack:
    0.0002     229968   1. {main}() /Users/whitebear/httproot/myapp/app/console:0
    0.0126    2063376   2. SymfonyComponentConsoleApplication->run() /Users/whitebear/httproot/myapp/app/console:22
    0.0144    2256968   3. SymfonyBundleFrameworkBundleConsoleApplication->doRun() /Users/whitebear/httproot/myapp/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:106
    0.1503   11751600   4. SymfonyComponentConsoleApplication->doRun() /Users/whitebear/httproot/myapp/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php:83
    0.1518   11753384   5. SymfonyComponentConsoleCommandCommand->run() /Users/whitebear/httproot/myapp/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:200
    0.1521   11758040   6. AcmeTopBundleCommandtwCrawlerCommand->execute() /Users/whitebear/httproot/myapp/vendor/symfony/symfony/src/Symfony/Component/Console/Command/Command.php:242
    0.2409   20673112   7. AcmeTopBundleCommandtwCrawlerCommand->initialRun() /Users/whitebear/httproot/myapp/src/Acme/TopBundle/Command/twCrawlerCommand.php:430
  101.5883  235069104   8. EntityManager52ee6dcbdb2bb_546a8d27f194334ee012bfe64f629947b07e4919__CG__DoctrineORMEntityManager->flush() /Users/whitebear/httproot/myapp/src/Acme/TopBundle/Command/twCrawlerCommand.php:284
  101.5883  235069200   9. DoctrineORMEntityManager->flush() /Users/whitebear/httproot/myapp/app/cache/dev/jms_diextra/doctrine/EntityManager_52ee6dcbdb2bb.php:402
  101.5883  235069280  10. DoctrineORMUnitOfWork->commit() /Users/whitebear/httproot/myapp/vendor/doctrine/orm/lib/Doctrine/ORM/EntityManager.php:389
  107.0744  259211408  11. DoctrineORMUnitOfWork->executeInserts() /Users/whitebear/httproot/myapp/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php:335
  107.1128  259277872  12. DoctrineORMPersistersBasicEntityPersister->executeInserts() /Users/whitebear/httproot/myapp/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php:952
  109.1604  267877360  13. DoctrineDBALStatement->execute() /Users/whitebear/httproot/myapp/vendor/doctrine/orm/lib/Doctrine/ORM/Persisters/BasicEntityPersister.php:279
  109.1604  267877408  14. DoctrineDBALLoggingLoggerChain->startQuery() /Users/whitebear/httproot/myapp/vendor/doctrine/dbal/lib/Doctrine/DBAL/Statement.php:161
  109.1604  267877408  15. SymfonyBridgeDoctrineLoggerDbalLogger->startQuery() /Users/whitebear/httproot/myapp/vendor/doctrine/dbal/lib/Doctrine/DBAL/Logging/LoggerChain.php:54
  109.1604  267877592  16. SymfonyBridgeDoctrineLoggerDbalLogger->log() /Users/whitebear/httproot/myapp/vendor/symfony/symfony/src/Symfony/Bridge/Doctrine/Logger/DbalLogger.php:80
  109.1604  267877592  17. MonologLogger->debug() /Users/whitebear/httproot/myapp/vendor/symfony/symfony/src/Symfony/Bridge/Doctrine/Logger/DbalLogger.php:102
  109.1604  267877640  18. MonologLogger->addRecord() /Users/whitebear/httproot/myapp/vendor/monolog/monolog/src/Monolog/Logger.php:423
  109.1604  267878968  19. MonologHandlerAbstractProcessingHandler->handle() /Users/whitebear/httproot/myapp/vendor/monolog/monolog/src/Monolog/Logger.php:244
  109.1604  267878968  20. MonologFormatterLineFormatter->format() /Users/whitebear/httproot/myapp/vendor/monolog/monolog/src/Monolog/Handler/AbstractProcessingHandler.php:35
  109.1604  267878968  21. MonologFormatterNormalizerFormatter->format() /Users/whitebear/httproot/myapp/vendor/monolog/monolog/src/Monolog/Formatter/LineFormatter.php:43
  109.1604  267878968  22. MonologFormatterLineFormatter->normalize() /Users/whitebear/httproot/myapp/vendor/monolog/monolog/src/Monolog/Formatter/NormalizerFormatter.php:40
  109.1604  267878968  23. MonologFormatterNormalizerFormatter->normalize() /Users/whitebear/httproot/myapp/vendor/monolog/monolog/src/Monolog/Formatter/LineFormatter.php:88
  109.1605  267881872  24. MonologFormatterLineFormatter->normalize() /Users/whitebear/httproot/myapp/vendor/monolog/monolog/src/Monolog/Formatter/NormalizerFormatter.php:70
  109.1605  267881872  25. MonologFormatterNormalizerFormatter->normalize() /Users/whitebear/httproot/myapp/vendor/monolog/monolog/src/Monolog/Formatter/LineFormatter.php:88

感谢您的评论,我知道它与$this-> em的内存相关

所以,我把代码设置为每隔5000次取消$this-> em;

if ($k > 1000){
    $this->em->flush();
    unset($this->em);
    $this->em = $this->getContainer()->get('doctrine')->getManager();
}

但仍然徒劳,我想我应该取消$this-> getContainer() – > get(‘doctrine’) – > getManager();

可能吗？或者我是对的？

解决方法

您可能需要阅读 Doctrine 2 Batch Processing.
他们正在使用这种模式：

// $em instanceof EntityManager
$batchSize = 20;
for ($i=1; $i<=10000; ++$i) {
     $obj = new MyEntity;
     $obj->setFoo('...');
     // ... set more data
     $em->persist($obj);
     if (($i % $batchSize) == 0) {
         $em->flush();
         $em->clear();
    }
}
$em->flush();

据我所知 – 在持久化之后,doctrine将所有映射对象保存在身份映射的内存中.因此,每个新对象的内存需求都在不断增长.如果你执行clear(),它将分离所有托管实体,如果你没有引用它们,它们将被垃圾收集.

注意：它将分离您正在使用的所有其他实体.或者你只能分离插入的对象$em-> detach($twNode); //从Doctrine分离,以便立即进行GC

编辑：在for循环后添加flush()以确保保存所有实体,即使实体计数不能被批量大小整除,如评论中所建议的那样

（编辑：李大同）

【声明】本站内容均来自网络，其相关言论仅代表作者个人观点，不代表本站立场。若无意侵犯到您的权利，请及时与联系站长删除相关内容!