I need to import 500k products very fast

Hello,

I migrated cs-cart on a 64GB RAM dedicated server with 12 CPUs and NVEe SSD. Even so ,several import sessions is taking around 24-30Gb of RAM.

I have a vendor selling around 500k products, I started the import via admin because ssh using cron script fail not been able to read the 900Mb file. After 3 days it grabbed 50K products and I had to stop the import due to some server php settings update. Since then , I’m unable to get any new products , I selected on import options to import only new products but it seems like cs-cart needs to check every single product that was imported before, so the number that the vendor has never changed, not a single product was added, I have no control over this , I can’t see anywhere what actually cs-cart does related to this import.

Can this be solved ? Is there any settings I have missed ?
Can 500k products be imported with other methods? because it seems like I will never be able to do it .
Is there any addons that really improved the import speed or they are all doing the same with only some extra import functions ?

My server is very fast , I have made all the required settings to enable cs-cart run faster (Redis, etc), I need to increase the import speed and to enable cs-cart to be able to read csv file so I can trigger the import via ssh using ‘screen’ , thi way I can atleast monitor the import, though the best thing will be for cs-cart or any other addon developer to make such option on admin area, some progress bar with import percentage ,also Pause and Stop buttons.