Advanced Import No Longer Works Using FTP Link

Bug tracker
1

I’m not sure exactly when this stopped working because I had it running as a cron but I am assuming either the 4.21.1 or 4.21.2 update.

My link is in the format of ftp://user:pass@domain.com/file.csv. When I try to manually run it from admin, it seems to be downloading the file because it’s a huge file and takes about 30 seconds. However, when that is done I am encountered with an " Error File can’t be loaded". There are no other errors that I can find in logs. I can still manually upload the file and run the import but now that I have done that, it will no longer save the ftp link because of the failure.

I have been trying to track down the issue but there have been so many changes in the associated files, I have been unsuccessful so far. I will keep trying to fix the issue because given how long bugs take currently, it may be years before CS-Cart fixes it.

In the meantime, I have had to create a script to run a cron to download the file and then run the cron for the import using the Upload a file method.

If anyone has any suggestions on how to fix it, I am all ears.

2

I experienced an almost identical problem recently. In my case, the message “File can’t be loaded” did not necessarily mean the CSV had failed to download. The failure occurred when CS-Cart tried to open/process the downloaded file afterwards.

There were two separate server issues involved:

Passive FTP ports were blocked by the server firewall/CSF
Once passive FTP connections were allowed, the supplier CSV downloaded correctly and the stock update worked.

On a password-protected development installation, CS-Cart downloaded the file but then attempted to load it through a URL. The server returned HTTP 401 Unauthorized because .htaccess Basic Authentication blocked CS-Cart’s own server-side request. allow_url_fopen was also disabled.

It may therefore be worth checking:

  • Whether a completed CSV actually appears in CS-Cart’s import or temporary-file directory.
  • The file’s size and ownership/permissions after the attempted download.
  • Whether PHP has allow_url_fopen enabled.
  • Whether PHP/cURL has FTP support enabled.
  • Whether CSF or another firewall is blocking passive FTP ports.
  • Whether the site or import directory is behind .htaccess password protection or another access restriction.
  • Whether the FTP password contains characters such as @, :, /, # or % that now need to be URL-encoded.

The 30-second delay could mean the file is downloading, but it could also be the FTP connection waiting until it times out, so checking whether the completed file exists on the server should help distinguish those cases.

As a test, I copied the downloaded CSV into:

var/files/exim/

and ran the same Advanced Import preset using the Upload a file source. That worked and updated 13,923 products, proving the CSV and import mapping were ok and narrowing the problem down to the remote-file retrieval/loading stage.

I m not sure that this was introduced specifically by 4.21.1 or 4.21.2, but the generic error hides the underlying FTP, HTTP or filesystem failure, so checking those items may reveal the actual cause.

3

Title: 4.21.2: FTP links in Advanced Import fail (“file can’t be loaded”) because Http::_request() now resets headers and getStatus() returns false for FTP

Version: CS-Cart 4.21.2, PHP 8.3

Symptom: An ftp:// file link in an Advanced Import preset fails with a generic “Cannot upload file / File can’t be loaded” error. There is no PHP error and nothing in the CS-Cart logs. The same URL downloads fine with command-line curl and with PHP curl. Debug output right after Http::get() showed status=false, a length equal to the full file size (about 152 MB), and no transport error.

Cause

  1. Http::getStatus() takes the status only from an HTTP/x.x NNN line in the response headers, and returns false if there is none. FTP responses have no such line.
  2. In 4.21.1 and earlier, Http::$_headers was never reset between requests. After an FTP download it still held the headers of an earlier HTTP request, so getStatus() returned a stale 200.
  3. 4.21.2 adds self::$_error = []; self::$_headers = ''; at the start of Http::_request(). FTP now gets false.
  4. fn_get_contents() in app/functions/fn.fs.php (unchanged) treats any status outside 200 to 299 as a failure, so false < 200 discards the complete download.

SUPPORTED_SCHEMES still lists ftp and ftps, so FTP is meant to be supported.

Workaround that fixed it for me: apply the status check only to HTTP(S) in fn_get_contents():

php

$scheme = strtolower((string) parse_url($path, PHP_URL_SCHEME));
if (in_array($scheme, ['http', 'https'], true) && ($status >= 300 || $status < 200)) {
    return false;
}

With only this change, setting up the preset with the FTP link, a manual import, and the cron import all work.

Suggested fix: either skip the status check for non-HTTP schemes, as above, or have Http report a success status for a completed FTP transfer. Other callers that check Http::getStatus() after fetching a non-HTTP URL may have the same problem. I haven’t checked.

Secondary issue: the file dialog on the preset edit page (File tab) doesn’t keep the FTP link, while the one on the Import presets manage page does, even though get_fields returns 200 with the full column list.

1 Like