php - Most efficient way to do batch INSERT IGNORE using doctrine 2 -

i have script needs go list of entries in database iterate on creating new entries in table if dont exists.

currently im doing:

foreach($entries $entry){     $newitem = new item();     $newitem->setattribute($entry->getattribute());     $entitymanager->persist($newitem);     try{         $entitymanager->flush();     } catch(\exception $e){         if(!strpos($e->getmessage(),'duplicate')){             throw $e;         }         $entitymanager = $this->getdoctrine()->getmanager();          //refreshes entity manager     }  } 

however doing way time intensive, there 1000's of entries , script times takes upwards of 10 minutes complete. have seen other posts suggest when doing batch processing flush every 20 or records problem if 1 of 20 duplicate whole transaction dies, im not sure how go , try , find offending entry exclude before resubmitting them again.

any appreciated.

you can 1 select fetch records exist in database, , later skip these records. additionally, try execute flush() , clear() once or play around batch size. suggest use transaction (if use innodb).

$this->_em->getconnection()     ->begintransaction();  try {     $created = array(/* primary keys exist */);     $i = 1;     $batchsize = sizeof($entries);     foreach ($entries $entry) {          if (in_array($entry->getmyprimarykey(), $created)) {             continue;         }          $newitem = new item();         $newitem->setattribute($entry->getattribute());         $entitymanager->persist($newitem);          if (($i % $batchsize) == 0) {             $this->_em->flush();             $this->_em->clear();         }          $i++;     }      $this->_em->getconnection()         ->commit(); } catch (\exception $e) {     $this->_em->getconnection()         ->rollback();     $this->_em->close();      throw new \runtimeexception($e->getmessage()); }