Jump to content

  • You cannot start a new topic
  • You cannot reply to this topic

How You Guys Manage Contents That Upload To Amazon Ec2? Rate Topic   - - - - -

 
  • iamau5
  • Newbie
  • Trial users
  • Join Date: 31-Mar 17
  • 7 posts

Posted 24 April 2017 - 04:10 AM #1

Hi all,

 

As you know that there is a chance that EC2 could destroy self-instance and recreate new instance from the image(image with site that already set up).

 

And when CS-Cart is launching. the framework only allow to upload incoming images to the host itself.

So there is a chance that when current instance is destroyed and recreate. The current content data from the previous instance will be gone.

 

So I want to know how you guys handle this issue. Is using cdn (cloudfront) help solving this problem? but I see that cloudfront still have origin on cscart host, and I'm not sure it will fix the issue. (I thought it will clone data to base on S3).

 

Or do I understand things wrong? please share how you guys fix this. Thanks in advance!



 
  • tbirnseth
  • CS Cart Expert
  • Authorized Reseller
  • Join Date: 08-Nov 08
  • 11023 posts

Posted 24 April 2017 - 08:39 PM #2

EC2 has caching issues as it relates to propagation delays.  I.e. FTPing a change to a file might take a couple of seconds or even a couple of minutes to be visible.

 

Turning off APC caching seems to help considerably.

 

Using cloudfront won't help you at all.  EC2 already does it's own cdn.


EZ Merchant Solutions: Custom (USA based) B2B Development, Consulting, Development and Special Projects (get a quote here).
Commercial addons, payment methods and modifications to meet your business and operations needs.


 
  • rawich_k
  • Newbie
  • Members
  • Join Date: 18-Jan 17
  • 10 posts

Posted 25 April 2017 - 03:55 AM #3

Hi tbirnseth,

 

Did you implement any solution to upload any static resources such as product's images or files to other data hosting server other than CS-Cart application itself? If you have done that before I would like you to share your idea.

For me, the possible solution that don't need to modify code at all is the network file system in Linux. Just mount the network path to /path/to/cscart/images and OS will do the rest.

 

by the way what is the service name of cdn in EC2?, its need to be installed?



 
  • tbirnseth
  • CS Cart Expert
  • Authorized Reseller
  • Join Date: 08-Nov 08
  • 11023 posts

Posted 25 April 2017 - 05:35 PM #4

I have not.  However, there are paid CDN servuces that will allow you to upload static data and then serve it from your site using Apache configuration controls.

 

I guess I'd ask "what is the problem you are trying to solve"?  Are access to your site worldwide?  What is the performance of serving your imagery from your storefront?  Are you sure you have a latency/distance related problem?


EZ Merchant Solutions: Custom (USA based) B2B Development, Consulting, Development and Special Projects (get a quote here).
Commercial addons, payment methods and modifications to meet your business and operations needs.


 
  • ThomH
  • Senior Member
  • Members
  • Join Date: 20-Nov 07
  • 1490 posts

Posted 26 April 2017 - 08:22 AM #5

Using cloudflare the static content will be cached automatically.

Do not believe everything that they are writing on their page (especially on free account), but may help in serving static content faster.


WebGraphiq offers a wide range of professionally developed, ready to use CS-Cart add-ons to provide additional functionality and boost your sales. The oldest active CS-Cart add-on development team. -- Since 2006 --


CS-CART ADD-ONS | FREE QUOTE | CS-CART DEVELOPMENT | @webgraphiq


 
  • iamau5
  • Newbie
  • Trial users
  • Join Date: 31-Mar 17
  • 7 posts

Posted 26 April 2017 - 09:24 AM #6

Hi tbirnseth,

 

We are not plan to make it worldwide yet, but the problem is. When EC2 reach it scaling somehow, it will terminate its instance and create new from image. And baam, all the static data on previous instance gone. That's the problem we are aware of.

 

So to prevent that, we want to make sure that situation won't happen by try to let our site point to other hosting server instead.

 

The paid CDN you mentioned could you suggest us which service do you use? And how to upload your static data there? As it there are some thumbnail images that will be generated through user clicking on site, will those images will automatically store on that server and act as an origin host too?



 
  • tbirnseth
  • CS Cart Expert
  • Authorized Reseller
  • Join Date: 08-Nov 08
  • 11023 posts

Posted 26 April 2017 - 06:18 PM #7

I don't use a CDN and I don't use EC2.  Neither are necessary for my business.  But services like Akami and real commercial CDN services can be configured to meet whatever need you have.  I'd just recommend staying away from those who want to capture your DNS (like cloudfare).


EZ Merchant Solutions: Custom (USA based) B2B Development, Consulting, Development and Special Projects (get a quote here).
Commercial addons, payment methods and modifications to meet your business and operations needs.


 
  • fleaffair
  • Senior Member
  • Members
  • Join Date: 11-Jun 14
  • 537 posts

Posted 15 June 2017 - 09:00 AM #8

Hi tbirnseth,

 

We are not plan to make it worldwide yet, but the problem is. When EC2 reach it scaling somehow, it will terminate its instance and create new from image. And baam, all the static data on previous instance gone. That's the problem we are aware of.

 

So to prevent that, we want to make sure that situation won't happen by try to let our site point to other hosting server instead.

 

The paid CDN you mentioned could you suggest us which service do you use? And how to upload your static data there? As it there are some thumbnail images that will be generated through user clicking on site, will those images will automatically store on that server and act as an origin host too?

There is an option in EC@ instance where it will not terminate itself , make sure you enable that.


Warm Regards,
Admin | MVE: 4.2.4

 
  • tbirnseth
  • CS Cart Expert
  • Authorized Reseller
  • Join Date: 08-Nov 08
  • 11023 posts

Posted 14 February 2018 - 10:53 PM #9

When you use https, the connection between the client and the CDN is secure, but the connection between the CDN and your originating site is not.

 

There are several other reasons why I don't recommend using a CDN at all unless you sell internationally (I.e. a reasonable percentage of your business is from far away).  

 

When doing development on a site that has CDN turned on, propagation delays can double or triple the cost of development by having to wait for whatever the current connection is to update from changes at the originating site.  Sites like cloudfare are slow to propagate which is one of the reasons they are free.

 

Those are a few or MY reasons.  If you're a US merchant and you sell mostly in the US, you would see very little benefit from a CDN.  It's when you start crossing oceans that delay can become a problem.....  There are no hard/fast rules.  Just do A/B testing to see how much benefit you get from additional managed services.


EZ Merchant Solutions: Custom (USA based) B2B Development, Consulting, Development and Special Projects (get a quote here).
Commercial addons, payment methods and modifications to meet your business and operations needs.


 
  • extrarius
  • Member
  • Members
  • Join Date: 05-Feb 14
  • 29 posts

Posted 15 February 2018 - 09:28 PM #10

We're actually hosting our site entirely within AWS ecosystem and have for several years. We do leverage Amazon CloudFront but only for static content such as product images and documents, javascript, css and custom content images (for static pages). The dynamic content is handled directly by the web servers which sit behind an Elastic Load Balancer.

 

To handle the javascript, css and content images we have Amazon CloudFront configured with a custom origin that points back to the Elastic Load Balancer on a non-default port that uses a virtual host only that serves the static content. These resources are kept in sync between the servers using GlusterFS. 

 

Product Images are handled with CloudFront pointing to EC2 servers running Thumbor for dynamic resizing and delivery. Documents are hosted on Amazon S3.

 

I should point out though that this setup is not a simple implementation, we had to make several key modifications to CS-Cart and the infrastructure requires several EC2 instances (2 just for GlusterFS alone) to make this all possible.

 

Even if you are only serving a specific set of countries (such as us which is only Canada and US) a CDN still helps offset the load of delivering static content with dynamic content.


HardwareAndTools.com

The Internet's Largest Hardware Store


 
  • tbirnseth
  • CS Cart Expert
  • Authorized Reseller
  • Join Date: 08-Nov 08
  • 11023 posts

Posted 15 February 2018 - 10:18 PM #11

You have done more work that most merchants are willing to do.  However, it certainly sounds like you thought through the problems have come up with solutions that address most of my concerns related to cloudfront and CDN's in general.  But as you say, this was not a simple task nor one that is easily implemented with standard cs-cart.  Good work.


EZ Merchant Solutions: Custom (USA based) B2B Development, Consulting, Development and Special Projects (get a quote here).
Commercial addons, payment methods and modifications to meet your business and operations needs.


 
  • extrarius
  • Member
  • Members
  • Join Date: 05-Feb 14
  • 29 posts

Posted 16 February 2018 - 12:40 AM #12

Appreciate the compliment, yes unfortunately tightly integrating CS-Cart to take full advantage of AWS was a lot of work but it has served us well. I'm trying to find ways to make that information translatable for people here to take advantage of but time is something I have very little of these days.


HardwareAndTools.com

The Internet's Largest Hardware Store