Hi All,
I know that CS has a .csv import/export feature. But, we need to import .xml files taken by the suppliers with some kind of automation.Price,stock info,Products,Departments, Product descriptions and so on.
Any possibility to do that? Or any addon available on the market?
Waiting for your earliest replies,
I’m faced with the same situation: 50,000 configurable products in an XML feed with suppliers, product option combinations, option exceptions, option images, quantity discounts, features, filters, etc - all updated on a weekly basis. I’m pretty sure this is beyond the capabilites of the CSV import tool.
My current plan is to use Altova Mapforce to push the data directly into the MySQL tables and then use Windows Scheduled Tasks to run that every Friday night. Does anyone have a better approach?
Glen
[quote name=‘sixnin9’]I’m faced with the same situation: 50,000 configurable products in an XML feed with suppliers, product option combinations, option exceptions, option images, quantity discounts, features, filters, etc - all updated on a weekly basis. I’m pretty sure this is beyond the capabilites of the CSV import tool.
My current plan is to you Altova Mapforce to push the data directly into the MySQL tables and then use Windows Scheduled Tasks to run that every Friday night. Does anyone have a better approach?
Glen[/QUOTE]
Thanks for your ideas Glen. We just need to find an easy solution for that? Maybe, someone in the forum know some kind of addon updating product catalogs from xml files.
Does anyone know?Please give us some feedback.
Sertac
Unfortunately, XML files have to be parsed and the tags applied to the proper DB tables and also the related tables. Each XML data file will be different and different rules need to be applied to it. XML is a great format for somethings and not so great for others. However, defining a standard XML format for importing data and then using XSL to translate XML data from a variety of sources to a common format would make this doable and reasonable.
However, getting anyone to agree on what fields should be supported is probably a nightmare.
Mapforce would still require you to map each XML tag to some kind of meta-data, then to map that meta-data to the appropriate DB field(s). That’s just the way this stuff works. Mapforce has put the effort into the admin interface and that’s where the value (and work) lies.
Doing a single XML->DB work is not that hard. What’s complicated and time consuming is providing the administrative interface to allow non-technical people to modify the configuration.
[quote name=‘tbirnseth’]Unfortunately, XML files have to be parsed and the tags applied to the proper DB tables and also the related tables. Each XML data file will be different and different rules need to be applied to it. XML is a great format for somethings and not so great for others. However, defining a standard XML format for importing data and then using XSL to translate XML data from a variety of sources to a common format would make this doable and reasonable.
However, getting anyone to agree on what fields should be supported is probably a nightmare.
Mapforce would still require you to map each XML tag to some kind of meta-data, then to map that meta-data to the appropriate DB field(s). That’s just the way this stuff works. Mapforce has put the effort into the admin interface and that’s where the value (and work) lies.
Doing a single XML->DB work is not that hard. What’s complicated and time consuming is providing the administrative interface to allow non-technical people to modify the configuration.[/QUOTE]
Tbirnseth,
Thanks for your great and detailed info about the issue.
I wanna get your ideas about: Is it logical to convert those xml files into .csv files in order to import them via Data Feeds on the back-end? Price, product descriptions, stock info, product pictures and other tags can be updated by using csv files?
To summarize it: 1.convert xml to csv 2. use data feeds to import that csv file 3. use cron jobs to automate it so,we can get rid of working on the database things,since CS has csv reader etc.
Waiting for everyone’s ideas about this post and for the main issue.Thanks.
Sertac
[quote]
s it logical to convert those xml files into .csv files in order to import them via Data Feeds on the back-end?
[/quote]
Sure. I would use excel to read the data from the XML and then write it as CSV. This would be the easiest way to work with it. But I’m not an excel expert so I can’t tell you how, but I know you can.
Pictures need to be FTP’d to your site. The import allows you to reference the path of the images, it does not contain the images themselves.
If you’re wanting it automated then you will not be able to use excel. You will have to write a routine that will translate the XML to CSV, using that output as the CSV file for a Product Import (not a data feed). Data Feeds are outbound entities in cs-cart.
What cs-cart needs is the ability to read an RSS feed (similar to Google’s Base/Merchant extension) that would allow it to update automatically when the RSS has new info. Kind of the inverse of what I do with my XML Data Feeds product (http://www.ez-ms.com/addonproducts/xml-data-feeds.html). But that COULD become a maintenance issue if/when the underlying database schema changes…
I’d be happy to develop an addon of the Product data feed if we could get an experienced set of users on the forum to agree to the fields to be supported and the tag names to use. A good starting place would probably be the CSV import definition with some adjustments Note that for large stores, this may not be possible due to timeouts and other limitations of the resources required to parse and manipulate large XML data files. Like stores with over a few thousand products.
[quote name=‘tbirnseth’]Sure. I would use excel to read the data from the XML and then write it as CSV. This would be the easiest way to work with it. But I’m not an excel expert so I can’t tell you how, but I know you can.
Pictures need to be FTP’d to your site. The import allows you to reference the path of the images, it does not contain the images themselves.
If you’re wanting it automated then you will not be able to use excel. You will have to write a routine that will translate the XML to CSV, using that output as the CSV file for a Product Import (not a data feed). Data Feeds are outbound entities in cs-cart.
What cs-cart needs is the ability to read an RSS feed (similar to Google’s Base/Merchant extension) that would allow it to update automatically when the RSS has new info. Kind of the inverse of what I do with my XML Data Feeds product (http://www.ez-ms.com/addonproducts/xml-data-feeds.html). But that COULD become a maintenance issue if/when the underlying database schema changes…
I’d be happy to develop an addon of the Product data feed if we could get an experienced set of users on the forum to agree to the fields to be supported and the tag names to use. A good starting place would probably be the CSV import definition with some adjustments Note that for large stores, this may not be possible due to timeouts and other limitations of the resources required to parse and manipulate large XML data files. Like stores with over a few thousand products.[/QUOTE]
Thanks for the info.Let me share an example of some suppliers’ xml file.
************************************************************************************************
-
2344
592
12252223536546
example product name
602
example brand
product info
0
4,1771
USD
18
small picture of a product
big picture of a product
************************************************************************************************
Of course, this is the simplest version of the xml file, there are some other codes to get the “product catalog”, “sub-categories” etc.But, in order to understand the basics,How can we make some kind of automation to get updates for these fields from suppliers to our CS store?
Thanks for your help,tbirnseth,
Sertac
[quote name=‘getreal’]Thanks for the info.Let me share an example of some suppliers’ xml file.
************************************************************************************************
-
2344
592
12252223536546
example product name
602
example brand
product info
0
4,1771
USD
18
small picture of a product
big picture of a product
************************************************************************************************
Of course, this is the simplest version of the xml file, there are some other codes to get the “product catalog”, “sub-categories” etc.But, in order to understand the basics,How can we make some kind of automation to get updates for these fields from suppliers to our CS store?
Thanks for your help,tbirnseth,
Sertac[/QUOTE]
That’s an another supplier xml file just for the prices.
*************************************************************************************************
−
607
12.0000
14.4000
USD
−
608
27.6000
33.1000
USD
−
609
14.4000
17.3000
USD
−
1301
41.5300
49.8000
TL
−
1302
41.5300
49.8000
TL
−
1304
*****************************************************************************************************
Sertac
In the future, please enclose things like XML docs inside of “code” tags so that indentiaton is preserved and things are easier to follow.
Using your info as a base, I would structure the XML this way:
main category
main category > next category
[quote name=‘tbirnseth’]In the future, please enclose things like XML docs inside of “code” tags so that indentiaton is preserved and things are easier to follow.
Using your info as a base, I would structure the XML this way:
main category
main category > next category
So, first of all , you would convert all of the suppliers’ xml files having different structure into these base above.Is that right?
1.How can you do that conversion?Manually for every xml?
2.After creating that kind of XML base file, is it easy to import into a back-end with some automation?
3.Now, the tags in the xml file and structure of the codes are clear,So, this can be done more easily or your past comments are still valid ?
Thank you,
Sertac
[quote]
1.How can you do that conversion?Manually for every xml?
[/quote]
No, you would need a conversion routine (probably XSL) for each input stream that did not conform to the model above. That’s where the data conversion sites make their money, they have built the tools and infrastructure to allow a customer to “map” the data to their needs as described in my first post.
[quote]
2.After creating that kind of XML base file, is it easy to import into a back-end with some automation?
[/quote]
After the data is in this form then the routines need to be developed to lookup the various products/features/options by name and to then update the appropriate database fields. Never said this was going to be easy or cheap! 
But it is all doable if someone’s willing to invest the effort.
There are some inherent problems in cs-cart where they use the ‘name’ to identify a resource but do not have a ‘title’ for display. So an option of Size in one product can be confused with Size in another that both use the name Size. That’s why you have to export products before importing them if you want options to “stick” because they prefix the otpion with the actual database key. But then this makes it impossible to use the data on more than one site. The options might need an additional attribute of ‘type’ for whether it’s a global option or local. Probably best way is to see if the product already exists. If so, and the option names are the same, just overwrite that option with the “value” data. Otherwise, create a new one.
[quote]
3.Now, the tags in the xml file and structure of the codes are clear,So, this can be done more easily or your past comments are still valid ?
[/quote]
Easier, not necessarily easy. There is probably 30 hours of work to get the basics setup. And then there’s probably 2-4hrs per different XML format to get the XSL built, tested and working.
I don’t think people would be willing to pay what it would cost to develop an addon of this nature. I usuallly price things as addons as 1/10 the cost to produce it. Hence I’d need 10 customer to just break even. Assuming there are 40hrs of work to get a single instance of this up and running, the cost (my cost) to develop is about $2200. Hence the addon would cost about $220 per licensed domain. If a group wanted to get together and pay for the development costs, then this could become the property of that group of people. I doubt most on this forum are willing to pay that kind of money, but the “real businesses” probably would because they understand that time is money.
I would be very interested in this. I know that my suppliers give live data feds fr stock but as for knowledge and implemeting this in CS …not a clue so something like what you are doing is very benificial to me.
Thanks
John
[quote name=‘johnbol1’]I would be very interested in this. I know that my suppliers give live data feds fr stock but as for knowledge and implemeting this in CS …not a clue so something like what you are doing is very benificial to me.
Thanks
John[/QUOTE]
Thanks for both of your replies.
I spent much time on searching this feature for Cs-Cart.Zen Cart,Magento,OsCommerce have that kind of feature while CS doesn’t. I will open that XML issue as a new idea on UserVoice.Would be true if I give this example from Magento?
[url]安全加密检测
The extension/addon has also XML profiler to revise different XML files before importing.
““Select the your xml file which is to be converted to Magento web store and provide the needed information in the input boxes, which is must for converting the data.
Select The XML file: Select your xml file.
XML Root Tag Name: The root tag name in the xml file in which the data contains. For Eg:’TemplateItem’
Magento Field Names corresponding to the XML tag: (Eg: sku,name,description): Specify the Magento database field name corresponding to tag names in the xml file.
Additional feild name(Eg:store:admin,websites:base): It is additional fields which are needed to add to CSV file which is must for Magento Data import and is not present in the xml file. The format should be like that ‘Magento fieldname: value;’(each data can be separated by semi colon). For E.g.: if ‘store’, which required field is not present in XML file, it should specified in the box. Because it is must for Magento product. In the above xml example some required fields in Magento is missing. So we have to add these through this box.
After filling data, and submit the form. It will convert the XML to CSV and will display the corresponding message. Then select the 2nd tab ‘Profile Actions XML’. Fill the form with corresponding details.
Profile Name: Enter the name of the profile, which used for further use.
Actions XML: It is same as xml which is used in Magento data Import.
Then select the 3rd tab ‘Run Profile’. Click the button ‘Run Profile in Popup’, it will execute data import process in a new popup window and will display corresponding message after completing the process.
Now you can check your Products database, the data in the xml will be imported to Magento web store. A history of profile actions to be maintained by using this module also. “”””
----Additional Features available with the Module----
Import simple products from the xml to Magento database
The data in xml file can be of any format
There is provision for mapping the fields in xml file to Magento database fields.
Data migration from other external applications to Magento web store.
Provision for customization of data import
Easy to manage
Thank you,
Sertac
Anyone there to help?
Thanks,
Sertac
[quote name=‘getreal’]Anyone there to help?
[/QUOTE]
Can only speak for me… The request is a lot of work and a rather large project. Unless someone (or group) was willing to pay for it, I’ve gone as far as I can go with providing info and context.
Note that from your info above that Magento is simply converting an XML file to CSV and than using their normal import mechanisms (what I suggested originally).
[quote name=‘tbirnseth’]Can only speak for me… The request is a lot of work and a rather large project. Unless someone (or group) was willing to pay for it, I’ve gone as far as I can go with providing info and context.
Note that from your info above that Magento is simply converting an XML file to CSV and than using their normal import mechanisms (what I suggested originally).[/QUOTE]
I see what you mean.
If we convert suppliers’ xml files into .csv files, then,how can we import that output on the back-end?And, can we make it automated if it is in .csv format?
Thanks,
We seem to be going around in circles.
There are two options:
- Create an automated mechanism that can read XML files from vendors, use XSLT to convert that data to a standard XML format and then have a process that can read that standard format and update the product inventory appropriately.
- Use a manual process (via Excel or other tool) of converting vendor XML to CSV, edit the CSV changing the column names appropriately for cs-cart and then using the cs-cart import mechanism to import the data. This could also be automated but you would have to have a dedicated method for each vendor format. You will probably not be able to do options and features effectively with this method.
#1 is custom development and the way to manage different vendor formats is via XSLT. If there are a number of different vendors, then this is the way to go since the cost is mosty in the initial development with a small cost for each added vendor.
#2 is the way to go if you’re doing it infrequently or only have a vendor or two.
#1 requires development and programming, #2 requires work and effort.
[quote name=‘tbirnseth’]We seem to be going around in circles.
There are two options:
- Create an automated mechanism that can read XML files from vendors, use XSLT to convert that data to a standard XML format and then have a process that can read that standard format and update the product inventory appropriately.
- Use a manual process (via Excel or other tool) of converting vendor XML to CSV, edit the CSV changing the column names appropriately for cs-cart and then using the cs-cart import mechanism to import the data. This could also be automated but you would have to have a dedicated method for each vendor format. You will probably not be able to do options and features effectively with this method.
#1 is custom development and the way to manage different vendor formats is via XSLT. If there are a number of different vendors, then this is the way to go since the cost is mosty in the initial development with a small cost for each added vendor.
#2 is the way to go if you’re doing it infrequently or only have a vendor or two.
#1 requires development and programming, #2 requires work and effort.[/QUOTE]
Tbirnseth,
Now,everything is definitely clear. Thanks for your kind responses.
Sertac
This X-cart addon looks like a good prototype for the addon we’d need.
[url]http://www.bcsengineering.com/store/product-importer-updater_pro-for-x-cart.html[/url]
[quote name=‘sixnin9’]This X-cart addon looks like a good prototype for the addon we’d need.
[url]http://www.bcsengineering.com/store/product-importer-updater_pro-for-x-cart.html[/url][/QUOTE]
Thanks for the prototype sixnin9.
I don’t understand how cs-cart owners update their inventory and communicate with their vendors without using XML’s. To do some kind of xml to csv conversion manually or calling the vendors for the product details are both not a good method at all.
Anyway,I created a new idea on User Voice,please give some support to my idea to get this xml feature integrated for Cs-Cart.
Thanks everyone for their support,
Sertac