There are three main issues that will affect import speed:
Server Resources: If your server doesn』t have enough available resources this will slow down your import. So if you see that your server is maxing out its CPU or memory usage during an import, upgrading to a more powerful server will help.
Database Size: WordPress stores data in a SQL database. When WP All Import is importing data, it does the actual importing by interacting with your database. The larger your database, the longer each interaction will take.
External Images: Downloading images from somewhere else during your import will always make your import take longer. If the images are large, or the server providing the images is slow, the impact will be even greater.
Specific Problems & Solutions
Large Databases, Bloated Tables, and Big Import Files
As your database grows in size your imports will take longer. It is not possible to avoid this slowdown because as your database grows larger your server and WP All Import must do more work.
For example, when checking for duplicates, WP All Import must search through more records. When updating existing posts with new data, the more posts on the site, the longer it takes MySQL to find the correct post to update.
As your database grows in size you will eventually run up against the limits of WordPress』 database schema. We have seen some users successfully import over 500,000 records into WordPress with WP All Import. And the sites they have created are indeed performant and stable. This, however, is the exception, not the norm. Those users spent weeks on their imports and deployed professional grade server optimization and caching to get that kind of performance out of WordPress and WP All Import.
▸ Solution
Sometimes your database is bigger than it should be. We』ve seen some users suffering from slow imports, and when we look inside their database we find that the wp_options table has over a million entries. The most common reason for wp_options bloat is the transients cache, which you can manage with a plugin like Transients Manager. Check with your host or a database professional to see if your database is similarly bloated.
If you』re trying to build a site with 500,000 products, you』ll need a very powerful and professionally optimized server. This isn』t to accommodate WP All Import so much as it is to accommodate WordPress and WooCommerce.
Typically on a shared host, somewhere between 50,000 to 100,000 records seems to be the upper limit. Anything beyond that often results in server timeouts, slow imports, and long load times.
Slow External Image Hosts
Many users choose to import images from external hosts. In some cases the images are quite large and/or the external host is slow to serve them.
▸ Solution
Run some tests with images disabled. If your import speed improves, try your import using the Importing Images From Your Server method so you don』t have to download the images to your server during the import.
Disable the do_action to Speed Up Imports
Every time WP All Import creates a new post it uses wp_insert_post. The problem with this function is that it wasn』t designed for creating a lot of posts at once – it was designed for creating one post at a time. It can be slow, and part of the reason for the slowness is the do_action calls.
▸ Solution
In Manage Imports ▸ Import Settings ▸ Advanced Options you can enable the Increase speed by disabling do_action calls in wp_insert_post during import feature.
This will prevent other software in your WordPress site from interacting with posts as they are imported, so other plugins that are supposed to automatically do things to new posts may not work for imported posts.
See http://codex.wordpress.org/Plugin_API/Action_Reference and http://codex.wordpress.org/Function_Reference/wp_insert_post for details.
Shared Hosts and Low Server Resources
All servers were not created equally. Many times hosting companies will cram as many sites as possible into their hardware to keep costs down. Sometimes they will limit the amount of PHP processing time you are allowed or the number of SQL queries you』re able to make. Or maybe they』ll throttle your processing power. Or maybe they don』t do anything, but the 1,000 other websites running on the same piece of hardware are consuming so many resources that your import has to fight for CPU time to finish processing.
▸ Solution
If you』re running low on available resources, upgrading to a server with more available resources (amount of RAM, number of CPUs, etc) will help. But it will usually only help if you』re running low on available resources. While the actual clock speed of your server』s CPU, RAM, and disk do play a role in how long your import takes, it』s not a very big one.
Think of it like a moving truck.
If you』re moving and your truck is too small you』ll have to make more trips and this will slow you down. Buying a bigger truck will help. But if your truck is already big enough to fit all of your stuff, buying a bigger truck (more CPUs, RAM, etc) isn』t going to help you move any faster. Upgrading to a server with a faster CPU and disk would be like putting a new engine in your truck. Sure, your move might go a little faster, but it should not be the focus of your efforts.
See our recommended hosts page for a list of web hosts that work great with WP All Import.
My Import Runs Faster On Your Server
Our server hosts hundreds and hundreds of WordPress sites and is running multiple imports from different users at the same time. The hardware we use is quite powerful. But, it』s likely that the testing installation is able to run WP All Import faster only because it has an empty database with very few plugins installed.
We optimized our server to make sure multiple imports could be run at the same time and to scale up server resources as needed. We also spent some time making sure people wouldn』t run in to any server errors during their imports.
We didn』t make any effort to optimize the server so it would run imports any faster. In terms of server optimization, all that you can really do is make sure you』re running the latest version of PHP and that you aren』t maxing out your server』s available resources.
Dealing With Long Import Times
Sometimes there』s nothing you can do to speed up your imports. In these cases you can structure your imports so that it doesn』t matter how long they take. If you set your imports to run automatically with a cron job then you won』t have to keep your browser open while they process – they will just run in the background.
If your site is suffering from performance issues while your imports are running, you can enable the Cron Sleep option in All Import ▸ Settings. This will slow down your import so that it won』t slow your site down for visitors.
