We need a QA group for testing Release Candidates - NOW

This is getting posted to CS-Cart as a ‘feature-request’ as well, but I would like to voice my opinion here:



We cannot have open beta version of CS-Cart at this late stage in the game. There are not enough technically minded people to support the releases such as what we have in 2.0.9 where the system is clearly failing in the face of real data and real business logic something has got to change.



The problem is clearly that the CS-Cart developers don’t have an adequate assembly of test data, or they do not have enough business experience to successfully put an e-commerce package through its paces. Sorry, that’s the stingy assessment.



The generous assessment is that they have the skills and the data, but lack the time and resources to conduct a true QA process with release candidates before they are released.



I will gladly take upgrades for the duration of my membership in an exclusive beta testing group that will run the upgrades across my data in development mode. I think it is safe to say that there would be other members of this board who also would consider doing this - perhaps free upgrades is not enough for them, but for me it would be acceptable.



The bottom line is, there are far too many customers who are not technically adept enough to deal with these upgrades - and frankly, they shouldn’t really have to if the bugs do not deal with code modification and template changes.



While I am technically minded, I need to know that my business is not going to have to disappear for a few weeks while bugs are worked out of software. I agree, this process does work to get bugs fixed, but getting cheap QA labor through inflicting buggy software on people who are NOT QA QUALIFIED DOES NOT WORK.



While labs is a step in the right direction, the sample data is not good enough to work with - it is trash, frankly, in that it doesn’t reflect real business logic or practices. Being able to offer and test sneakers in different colors is ELEMENTARY. It bears little resemblance to the complexity of running a business, managing inventory, marketing, promotions, etc, etc.



There needs to be an RC phase that gets released to a limited group of people who understand QA procedures and can develop a policy. Otherwise you will lose customers regardless of how good the software is eventually.



Case in point, we are about to move from a $150,000 operation to merge into a $500k+ operation. We are expanding our services to include phone ordering, fax ordering and discounts for internal departments, bookstores and parishes. It would be impossible for a group of software programmers to ‘thinktank’ the realities that our business is facing.



So now, after doing prep work and getting ready to launch discount pricing tiers for the holidays, I am forced to contend with errors in pricing because this software update was not tested against real business logic.



Don’t get me wrong, I know software has bugs that need fixing, and I expect that - but it is unfair to unleash that on non-programmers who are just trying to keep their site from crashing so they can stay in business. Frankly, I don’t expect to find as many critical bugs as I (and others) have uncovered in this recent release.



My challenge - create a group of people who have real business data and technical skill to run the Release Candidate update process prior to public release. Give them whatever incentive you find reasonable, but do it soon because if this happens again there is not going to be an opportunity to make up the loss.

I agree wholeheartedly with what you said …



Coming from the perspective as top recognized systems and security expert, I sit and look at most of the various posts here and most of the discussions and complaints are all on meaningless non-sense stuff that would take someone like myself 2 seconds or less to sort through but most users out there aren’t “experts” and CS-Cart should not be releasing scripts meant for production that require users to have an expert background because they simply failed to follow through with proper testing protocol in development.



The other thing that is quite annoying is the on going blind assumption that all users are using Apache Module (DSO) PHP based systems when the vast majority is SuPHP presently, with some phpSuExec, and the actual remaining DSO based systems a minority that is waning very quickly. It would be nothing for them to put switches in their code to handle the different systems and set the correct permissions for each. In fact, php actually has a built in function for this purpose (php_sapi_name) and would be an easy brainless update to the code.



(I should know given that I’ve already patched hundreds of these)



Cpanel has 5 development trees – BETA, EDGE, CURRENT, RELEASE, and STABLE and new code works it’s way down through those channels.



Why not do the same for CS-Cart?



At the very least do some more bloody ‘beta’ testing before release!



I can fix the bugs in a heartbeat blind folded but the average user is not going to know how to do this and is going to be left with their store broken, have extended downtime, and hire someone like myself with each upgrade just because thorough enough debugging wasn’t done before production release.



I was reading over a number of the complaints on 2.0.9 yesterday and noticed a common factor between many of the issues — most are related to the exact same set as the old PHP 5.1 to PHP 5.2 migration checklist which makes me wonder if they are only reviewing code on one system and an outdated one at that!



For now though I agree with you and I guess for the moment, it’s left for the “experts” to help all the “non-experts” with every single install though that shouldn’t be necessary especially coming from a commercial application deployed for general public use! :roll:

Yes, and I think the telling part of your analysis is that you work in the computer field. If you review the wide variety of sites hosted on cs-cart, you’ll find that the majority of the folks who use it are not even selling electronics or technology products.



Point being most folks who run a cart are not interested in learning to be programmers. They are trying to run a business doing what they do best for a living. Software like CS-Cart does a good job of making this happen with minimal effort when it works well and has a stable run - but when it doesn’t the typical user does not have the ability to determine a) what the problem is, b) how to report it, c) how to get their business back in its feet.



If you observe how many people have stuck with their 1.3.5 installations, these are the folks CS-Cart was designed to help and serve - the business owner who is not really a programmer.



For me, I am both, but trust me - I would much rather be doing one or the other - I wear about 4 hats as it is in my job and adding debugger to the list is not my favorite thing to do.



I have considered in off-hours to analyze the system closely - but here again it becomes difficult to do this and feel like I am on my own in the process. I believe Sno has pitched the idea of having a sort of developer forum, but again… coders who try to pull things together that are new and innovative - that becomes a frustrating endeavor when the code base you are trying to build new tools for is unstable.



Most shopkeepers are patient when it comes to new features and impatient when it comes to bugs that bring down sales or damage their store’s reputation for quality and usability. While I realize bug squashing is not fun for programmers, it is even less fun for shopkeepers who are already paying for the privilege.



I do think some sort of developer SVN tree as you mention above might be helpful. It would at least allow some additional diagnostic feedback for CS-Cart.

Agree with all the above.



Apparently, we need to resurrect the “Listen to your customers” post in Bug Tracker since they seem unable to figure any of this out on their own>



To CS-Cart: “Can you hear us now?”



Bob

I agree and below is the text of my “Bug Report”



Please add your comments



CS Cart developers are not testing 2.09



We love CS cart and want you to do well and make lots of money



While you may think that you are testing your software your customers do not agree and it is driving us crazy with frustration and lost sales.In the modern business world it is your customers whom define what a bug is not the developers.



You need to deliver bug-free software with clear comprehensive documentation - end of story.



If need be you should work 7 days a week long hours to fix the problems.



Not commenting on the frustrations of your customers as posted in these forums is not acceptable.



At a minimum a short post of - we are sorry we made a mistake and will work 7 days a week to fix the problems is required - and then give us regular updates.



And respond to our questions publicly.



Please see this thread as an example:



[url]http://forum.cs-cart.com/showthread.php?p=64207#post64207[/url]

I agree completely with the concept of having a QA test group if it may prevent the likes of 2.0.9 occurring again.



Whilst I would not call myself “techie” I have managed to do upgrade successfully, install and slightly “mod” 1.3.* though to 2.0.8 - then came 2.0.9 which failed miserably and put a huge dent in my confidence (in my abilities and more importantly in CSCART).



Us “shopkeepers” look forward to improved functionality etc and the fixing of bugs, but when new versions of software hurt your business, especially in these hard times, you begin to wonder why you bother. In fact I now find myself looking at alternative carts in the hope that the grass is greener over there. Many of the carts out there now are catching up with CS in terms of functions, and even some of the free ones are!!



I don’t want to leave CS and would love to see it move forward but they need to listen to what the customer says. How many times has it been posted “fix the bugs before you add anything new



Anyway, let’s hope a lesson has been learned and this situation very quickly gets sorted out.



Back on the subject of QA, although not techie, I am very familiar with the cscart functionality and would certainly be willing to assist with QA in terms of the backend functions and storefront.

I’m working on management and I’ll report back with where I get.

I don’t know what to say other then from the very beginning, which dates at least 4yrs back, CS has never asked for my opinion on new features and the usability of new features despite the fact I repeatedly keep asking them to ask me questions about the new features they have coming, so they can at least be coded from a shop owners point of view. I gave them the products tabs mod, they should have gone with what I had, it was tested by CS users with rave reviews for the last 2 yrs, then when they finally decided to put it in, they completely left out the global tabs which was the most important feature of the addon.



It’s like giving someone a Ferrari for free but without the steering wheel, what are you going to do with the Ferrari now?



I know their is alot of BUG issues but their are also alot of USABILITY issues that hinder the cart, just to name a few are:



Still to this day without modifications I still cannot view which products a customer ordered on the order list page without clicking open every single order, this takes forever. There should be the “extra” expand/collapse link to open and view the product data, I have already done this as a mod, but I shouldn’t have to do this every release.



There is no way to search for banners or filter them with the saved search functionality. Many of us create lots of banners for every season and holiday and event, I’ve asked for this feature forever, and have since had to code it myself.



I think a QA Group outside of CS is a good idea, however #1 their going to have to compensate us somehow, either through significant discounts on support, development and licenses. Otherwise it is hardly worth one’s time, and to be honest even if they gave us everything for the free that would only touch the frosting on the cake in terms of the ROI they would receive back from the QA Group. CS you should very seriously consider this QA Group, however you have to understand this. If the QA Group tests a new release and decided certain functionality needs significant changes, and completely re-worked, then the importance of the QA Groups’s decision is above yours, and then the QA Group’s decision’s should be implemented.



You don’t see the QA Group at Boeing telling the engineers they need more rivets on the wings or the wings can potentially fail and then have the engineers coming back saying everything’s fine, we’re not going to change anything, it’s designed as functioned? what?



There are literally hundreds of little USABILITY issues along with the bugs that need worked out - Sno

[quote name=‘snorocket’]I don’t know what to say other then from the very beginning, which dates at least 4yrs back, CS has never asked for my opinion on new features and the usability of new features despite the fact I repeatedly keep asking them to ask me questions about the new features they have coming, so they can at least be coded from a shop owners point of view. I gave them the products tabs mod, they should have gone with what I had, it was tested by CS users with rave reviews for the last 2 yrs, then when they finally decided to put it in, they completely left out the global tabs which was the most important feature of the addon.[/QUOTE]

The developers would spend a lot less time fixing things if they considered input from their customers (you know, the people who actually run an e-commerce business) in the first place. Obviously, customers will sometimes be blinkered by their narrow interests and thus miss out on the “big picture” but using the input from customers (especially as it relates to business processes and usabilty) would be a HUGE win for both CS-Cart and its customers.



Bob

[quote name=‘Traveler’]If need be you should work 7 days a week long hours to fix the problems.[/QUOTE]

As I have commented elsewhere, they should not schedule a release at the end of the week knowing in advance that there will be no support available for a minimum of 48 hours. Even if the bugs were minimized by thorough beta testing, having quick turnaround on small bugs immediately after release is imperative.



Bob

I do agree, releases should ONLY be scheduled on Mondays. I work in IT, and we NEVER do anything major after Wednesday unless we plan on being around ALL weekend.



It’s just part of the IT culture.

Actually, nothing wrong with a Monday release if the suport staff will be around all weekend.



24 hour/365 day service is a must also.