4Gb Memory Limit Exceeded Importing 768 Products

Exported products from a production site (4.2.4) to import into a new 4.3.6 site.

Upon import, it runs for a few minutes doing "Updating links"... then process terminates with this error in the error_log (and of course, the progress box is still saying it's working)


[18-Apr-2016 13:40:46 America/Los_Angeles] PHP Fatal error:  Allowed memory size of 4294967296 bytes exhausted (tried to allocate 72 bytes) in /home/calzini/dev436/app/Tygh/Database/Connection.php on line 852

So what is it about 768 products that is chewing up so much memory? I had exported all fields in the default product export.

I tried increasing to 8GB and it does the same failure but with the allocation amount at 8GB instead of 4GB.

Any known issues with import on 4.3.6?

That's a lot of memory. I posted a similar issue in bug tracker and basically CSC said that they needed to figure out how to break the large files up into smaller files.

Edit: http://forum.cs-cart.com/tracker/issue-6090-php-error-on-large-imports/

Yeah, but 768 products (rows) isn't exactly a large csv file.

It probably depends on size of file and not number of products. You said you were exporting/importing all fields so I imagine the size of the file was large.?

1.6MB filesize, 782 rows.

Not large to me and it was being imported from 'server'

Nope, doesn't sound like the same issue as mine. I would guess it has something to do with server because I import/export large files all the time, without major issues, running only 512mb memory allowed. Even in the bug report I reported, the import completes.

There are not any other errors associated?

I removed all non-ascii characters from the data and still no joy.

I was able to trim it down to a single data line that chewed up all the memory. But I can't see what might be causing the issue.

I submitted a bug and gave them the 1 line csv so hopefully that's solving enough of the problem for them to find root cause.

That there is no logging and no error reporting is a real pain. It all becomes guess work.

And I do notice the response from CSC here too.

I didn't see the file in bug tracker. Did you upload it?

Yes, but I believe uploaded files are hidden from the submitter and the user. And so far, no response on the bug report.