Jump to content

  • You cannot start a new topic
  • You cannot reply to this topic

Advanced Import Csv Extremely Slow Rate Topic   - - - - -

 
  • vin86
  • Member
  • Members
  • Join Date: 16-Nov 09
  • 40 posts

Posted 12 April 2021 - 07:13 AM #1

Hi,
i have cs-cart 4.11.3. I must import a big csv file with 180000 rows. Import with advanced import it's extremely slow (i must update only availability of products, for this reason i select only product code row and quantity).
 
To import this file i spent a lot of time (generally 1.30H). There is no way to import this file faster?
 
I have a dedicated server with Intel Xeon with 10 cores, 64GB of memory and SSD.
 
Thanks for support


 

Posted 12 April 2021 - 08:50 AM #2

If you like to extremely fast import of products availability please have a look our addon

 

Our addon was optimized for large XML and CSV files.


Only helpful and unique addons https://store.retailfactory.ru


 
  • vin86
  • Member
  • Members
  • Join Date: 16-Nov 09
  • 40 posts

Posted 12 April 2021 - 09:01 AM #3

If you like to extremely fast import of products availability please have a look our addon

 

Our addon was optimized for large XML and CSV files.

 

Thanks, but i want to know if it's possible resolve with Advanced import of cs-cart



 
  • tbirnseth
  • CS Cart Expert
  • Authorized Reseller
  • Join Date: 08-Nov 08
  • 12073 posts

Posted 13 April 2021 - 10:49 PM #4

Does your csv only contain those columns?  Or are you providing all data and telling the mapping to only utilize certain fields?  If the latter, you might try getting rid of all the unnecessary columns and see if that helps.


EZ Merchant Solutions: Custom (USA based) B2B Development, Consulting, Development and Special Projects (get a quote here).
Commercial addons, payment methods and modifications to meet your business and operations needs.


 
  • harmsmitsdev
  • Senior Member
  • Members
  • Join Date: 22-Feb 20
  • 183 posts

Posted 14 April 2021 - 07:32 PM #5

Does your csv only contain those columns?  Or are you providing all data and telling the mapping to only utilize certain fields?  If the latter, you might try getting rid of all the unnecessary columns and see if that helps.

Everything is done iteratively in CS-Cart, not collectively / in batches. So everything requiring a lot of iteration is just a pain. Now, if you have 180000 lines, its indeed going to take a while.


harmsmitsdev@gmail.com


 
  • tbirnseth
  • CS Cart Expert
  • Authorized Reseller
  • Join Date: 08-Nov 08
  • 12073 posts

Posted 14 April 2021 - 08:04 PM #6

My point was that if he has 100 columns in his data and is only using 5, reducing the number of columns COULD help performance.  It will certainly cut down on memory.  Yes, 180K rows will take a while no matter what.  But if you can reduce the memory requirement by 90% you might find that it actually runs faster.


EZ Merchant Solutions: Custom (USA based) B2B Development, Consulting, Development and Special Projects (get a quote here).
Commercial addons, payment methods and modifications to meet your business and operations needs.


 
  • vin86
  • Member
  • Members
  • Join Date: 16-Nov 09
  • 40 posts

Posted 15 April 2021 - 05:38 PM #7

My point was that if he has 100 columns in his data and is only using 5, reducing the number of columns COULD help performance.  It will certainly cut down on memory.  Yes, 180K rows will take a while no matter what.  But if you can reduce the memory requirement by 90% you might find that it actually runs faster.

 

file has only three columns. for import this file faster, i have split this csv in five parts and import in the same time. it's only solutions to resolve this problem. i have contact cs-cart but i have to pay to have support.  :-(