harmsmitsdev

Hello,

I am thinking of opensourcing an ElasticSearch integration I have been tuning for a while. I would like to get your feedback so I know how I should be offering this product. I do not have enough information with regards to what I should expect in terms of support queries, possible infrastructure changes, etc.

The integration currently works completely with product faceting (using the CS-Cart UI), typeahead search, and dynamic product faceting (search page). It also supports multi-store, multi-vendor, and dynamic customer prices (to a limit of 50, although this can be changed in ElasticSearch).

Please answer the questions in the polls.

Thanks in advance!

Kind regards,

Since end of 2020, I have been working on bringing some new features to CS-Cart myself. One of the things every developer should know of, which can greatly help, is `multiquery`. MultiQuery is a concept where one can send multiple queries in one go to the database.

Why should you use multi query? Well, in some *larger* environments, there simply is a round robin time for queries, which can easily be as high as ~.5ms. Now, Cs-Cart by default requires about 60 queries for a normal page. A product page will take up a hundred or so. MultiQuery offers a partial solution in this case, where it will allow you to send multiple queries at once to the database.

Its really easy to use, and only requires people to have defined the 'Mysqli' backend for their database (NOT PDO). Over the next few months, I will need to spot some areas where these things can be used, but one that you will immediately see is in `fn.images.php`; Usage is as follows:

list($query1, $query2) = db_multi_query([
    [
        MultiQueryTypes::ARR,
        'SELECT * FROM ?:products LIMIT 5',
    ],
    [
        MultiQueryTypes::HASH,
        'order_id',
        'SELECT * FROM ?:orders LIMIT 5',
    ],
]);

`\Tygh\Enum\MultiQuerTypes` holds all available types of querying in CsCart, think about `db_get_row`, `db_get_field`, `db_get_fields`, `db_get_array`, etc. The `db_multi_query` function will keep its array indices, meaning you could also directly assign it to the view controller with predifined offsets if you really want.

You can dynamically construct the array to bundle a lot of common queries if you prefer. But please mind, this is no excuse for bad queries. There are simply scenarios where one will require a variety of queries, since a UNION won't work, and that is what this is intended for.

In 4.14 I might add 'async' functions (if time permits) for this as well, where it will execute a stack of queued functions once a result is resolved, or when a blocking query is executed. This should greatly help to get some better performance for environments with high round robin times, especially since you can lift on queries CS-Cart already sends.

Anyways, I thought I'd bring this to your attention since most of this stuff never makes it into the release log.
 

Hello,

After having used CS-Cart for over 6 years now, I have gathered quite the experience, and have always paid attention to performance and all. Last year, was a turning point for our own website, where we invested in our entire own frontend (without *any* framework). This means, no Vue.js, no react, no jquery, nothing along the likes of that.

Now, bear with me, this is quite a technical story, so don't get scared if you don't fully understand it. If you are not interested in the technical part, you may skip the first 3 chapters.

Why re-invent the wheel?

This move came with quite a number of challenges, We have since created a custom framework (https://github.com/b...ands/elementals), which allows us to create lightweight, lazy and responsive components (meaning that e.g. mobile components are not loaded on the desktop) and vice versa. It also allowed us to bind the aforementioned components to CS-Cart blocks and also to define the scope of components (bound to a block, bound to a dispatch, or just plain global).

Now, this is what you would normally see when using e.g. Magento, where you can use components and load components only when you need them, instead of combining them into one huge blob of js / css (which is what CS-Cart currently does). This will usually result in e.g. pagespeed complaining that you use only ~15% of the CSS you deliver.

Bundling

To minimize these resouces, pagespeed recommends to split your code into different bundles, this is what is called 'bundling'. To read more about this, check out this article: https://medium.com/m...rs-f2db1adad724. It basically avoids sending a huge chunk of unused code, by splitting the code accordingly, and only loading the additional code when it is required. It is self-explanatory that this of course, improves your page speed ratings tremendously.

But how can you bundle in CS-Cart accordingly? Well, remember we defined our components by blocks in the block manager and some other criteria? This allowed us to pull of quite some amazing things. We have created build scripts, that will interface with the block manager, and build an optimized version of your store and bundle assets per dispatch. This, along with the rewrite of the entire theme, presented us with some truly astonishing numbers.

The default CS-Cart theme, has about 1.07MB of JS (including carousel) and 390KB of CSS. Now, this is without any modules or themes. Taking a look at a theme like UniTheme will give you even worse numbers. UniTheme weighs in with 1.14MB of JS and 690KB of CSS. Obviously, its pretty hefty and for good reason page speed complains about this. By using block-level optimization, we ended up using only about ~270KB of JS, and 70KB of CSS.

By doing some very basic math, we can see that already our store is about 80% less heavy on resources than unitheme. There are two factors which result in this huge difference. The first of which is code bundling. Unitheme loads *all* assets in advance. Even the ones you do not need, or might not need at all. Secondly, the usage of libraries and just *adding* code plays a big role. Unitheme does not remove any code, instead it only adds more and more code. It is because of this very reason, why we opted for a complete rewrite from scratch.

Rendering optimizations

CS-Cart, by default, does a poor job of optimizing the rendering time. This is by far the most important aspect of any store. There are 3 things one should account for:

1. How fast a user can see something. This is also called the time till the first contentful paint. It is the time it takes for a user to e.g. see the image and content on your product page.

2. How long it takes for the page to become interactive. In other words, how long does it take before e.g. the add to cart button works.

3. Total blocking time. This is usually the biggest factor. This is the time a user is *unable* to perform any actions on your website due to fetchign resources, rendering the page, parsing JS, etc.

Here are some numbers to take into account:

CS-Cart defaut:

First Contentful Paint: 3,4 S

Time to Interactive: 19,1 S

Total Blocking time: 2,42 S

Unitheme:

First Contentful Paint: 1,7 S

Time to Interactive: 11,3 S

Total Blocking Time: 2,46 S

Our store (with Google Analytics and Facebook SDK):

First Contentful Paint: 1,1 S

Time to Interactive: 7,2 S

Total Blocking Time: 0,2 S

Now, you might ask, wtf is that blocking time? Well, there are hundreds of tiny optimizations one can do, to avoid blocking the page rendering (which takes quite some time to figure out), but I will share some common ones with you.

1. Inlining assets. Eventhough people might condemn inline CSS / JS, they are a fantastic way of reducing the blocking time. If you recall correctly, we mentioned CS-Cart basically throws all CSS together into one big chunk of text. The biggest problem for rendering, is that if you open the webpage, you will first have to load the CSS sheet (which is a blocking operation), which takes a lot of time as it just takes time to fetch. Instead, we inline a stylesheet which contains ciritcal CSS for rendering the application. This means the browser won't require to fetch any CSS assets in when rendering the page (thus not blocking the application).

2. Web fonts. Most people might think, what might fonts cause? Well, they also cause a render block. Meaning that if you add fonts (e.g. fonts that include icons, like in cscart and unitheme), it will block the rendering process if you have to load them. Usually this doesn't happen as fonts in e.g. fonts.google.com are already installed on your system, but fonts that include icons usually are not. This results in an additional block of rendering of your application. Instead, we opted for lazily loading fonts (and flashing a user installed font first). This can be done by using e.g. https://fontfaceobserver.com/

3. Layout shifts. Every time you load a stylesheet, and your content somehow shifts a tiny bit, your webpage margins / paddings will have to be recalculated. This again results in blocking operations.

By optimizing the aforementioned 3, along with a hundred or so other areas of interest, we managed to pump out an almost non-blocking page render. Along with a 100 mobile score on google pagespeed, which is unprecedented for e-commerce websites.

What now?

Like mentioned, I invested a great deal in this. Just to squeeze out that final tiny bit of performance. Now, before you say, how do I do this myself, you will need to know about 3 things that will certainly cause some issues for most you.

1. Changing the location of a single block, if it has script dependencies that are not parsed, will require you to rebuild all your assets, which takes, a bit of time. Our webpack builds take about 500 seconds on average (without any caching). These have to be triggered manually.

2. Third party addons will not work on the frontend. All styles have been reworked, the grid is different, everything is different. Addons will have to be adopted to work with my theme.

3. Everything is written in PHP 7.4, as I like strict typing. It is not backwards compatible with PHP 7.3 or below.

So it certainly is not for people that want to use the block manager for comfort. I can not make this generally available as of now, without some serious more efforts, that avoid people form having to touch the CLI.

Planning to sell?

Eventually, I plan to distribute the theme at a slow pace. As of right now, I consider it to be only fitted for our website. It is simply too specific to distribute as of right now. I do have some questions for people just so I will hit the nail on the head:

1. What would you consider a valid price for a theme like this?

Currently the line of code count stands at a solid 165K+ and the theme does not use *any* of cscarts frontend code base.

2. What are some *required* features?

It currently includes a product accessories addon, an analytics addon (which is able to track users with UTM tags), a deal of the day module, advanced banners, cron managent, report-uri integration, frontend error detection (which logs browser errors to the server), product videos, multi-store seo (which makes href-lang tags across linked multi-stores) and custom product stock statuses.

Also, for Dutch customers it includes a bol.com, amazon, sendcloud and exact integration (for a premium obviously).

Furthermore, it also has completely rewroked email templates based on Inky. In short, all email templates / notifications have been reworked to be responsive and be supported accross many different devices.

3. Is monthly billing a problem?

I am strongly considering monthly billing, as first of all, api's from exact online / sendcloud / bol.com / amazon, are continuously changing and require my attention at least once a week to make sure everything remains up to date. Furthermore, I will provide my software as is, bugs in older versions will not be fixed for 'free'.

I will also be able to provide consistent updates for *all* modules, these will all be part of the monthly fee.

4. Do you mind that for *every* third party module that does something for your frontend, you will need it to be adjusted for you? If so, what are respectable timeframes?

5. Since I am sing AWS for almost our entire infrastructure, I recommend people to move to at least something similar. Would it be reasonable to require either AWS or Google Cloud for your hosting?

Final words

I will share a test instance in the near future. For now, the domain of our store, can be provided as per request.

All code will be encrypted, unless a non-disclosure agreement is signed. This is a way of protecting our source code from illegitemate practices, but also to protect myself from the fact that other companies might copy our codebase

Hello,

About one and a half week ago, an SQL injection vulnerability located in the `Product of the Day` add-on caught my attention. Since then, the according changes have been made to the `Product of the Day` add-on from AlexBranding. Therefore, it is of the utmost importance that you update the `Product of the Day` addon.

My research has shown that this vulnerability has been present since at least version 1.4, therefore all those affected are urged to either disable/update the addon or contact AlexBranding at the earliest convenience.

Exploit details and research will be published later next month to give all those affected a fair chance at patching their vulnerable software.

Kind regards,