I’ve setup data feeds in Magento before, but not for a while and I’ve generally used custom extensions or asked developers to handle the majority of the setup. I recently needed to setup data feeds for a client (Magento 1.9 CE) for their PLA campaigns and to use across various aggregator websites (Style.com, Polyvore etc), so I thought I’d write a blog post about the process.
To provide some background, I operate as an eCommerce consultant for the client – I’m not a developer, more of a solutions consultant. I opted to use the Wyomind extension because I’ve used it before and it’s a very manageable route for data feeds, as opposed to some of the other extensions which create a single feed designed only for Google Shopping. This extension is also a bargain at only 95 euros!
Out of the box, Magento doesn’t have an easy method for exporting products in a format that could be used for feeds. A third-party extension is typically used to bridge this gap, and Wyomind is considered the de facto standard for most Magento retailers. Wyomind actually offer two extensions: Simple Google Shopping (specifically for merchant centre) and Data Feed Manager (more flexible around the types of feeds being created with lots of pre-formatted feeds). However, since Data Feed Manager does everything that Simple Google Shopping does, as well as supporting a further 18 other shopping feeds, it makes sense for most retailers to choose Data Feed Manager (it’s also only marginally more expensive).
What is a data feed?
Put simply, a data feed, or product feed, is a file containing all of your products’ information, such as name, description, price, availability, size / weight, manufacturer etc etc. The file is typically created in .txt, .csv or .xml format, and sent, on a regular basis, to shopping channels (such as Google Shopping, Amazon, Polyvore, Lyst etc) or resellers. When submitted correctly, your products should be displayed on the channel in question, whenever a match is found when a visitor to that site searches for a product. Some channels are free whilst others require payment, typically on a per-click or a commission basis.
Which feeds are supported?
Wyomind’s Data Feed Manager is flexible enough to be able to support any data feed requirement, and comes with 19 templates pre-installed, to cover the most common shopping engines. These include Google Shopping, Amazon, eBay, Kelkoo, Pricegrabber, Nextag, Shopzilla, Shopping.com, Bing Shopping and more. Whilst some shopping engines will prove more useful to a UK-based merchant than others, lots of merchants will want to target as many channels as possible to acquire customers that wouldn’t have found them previously. The cumulative effect of even a low levels of sales coming from these shopping engines can make a big difference to the bottom line and it makes sense to use them with the low overhead with the feed and the CPA-based option available in places.
Licensing and installation
Wyomind has taken a pragmatic approach to licensing, whilst also building in checks to prevent unlicensed use. As with any new extension, it is sensible to test it thoroughly in a staging environment, before moving to your production environment. Unlike some extension suppliers, Wyomind is perfectly happy with this, and permits use of the extension on both staging and live environment URLs. An IP block mechanism is in place however, to prevent misuse across multiple URLs.
Installation of the extension is quick and easy, but do remember to disable compilation prior to installation, if you are using it. You can safely re-enable compilation once the extension is installed. If installing extensions fills you with dread, you can always opt for the professional installation option when purchasing the extension. For just $30 extra, Wyomind will install the extension for you, and check it thoroughly before handing it over to you.
Once the extension is installed, there are a few general configuration options that need to be set, before moving on to set up the first product feed.
Many of these core configuration settings are already set by default, and can be left as they are initially. However, you may want to consider switching ‘Include disabled products’ to No, as it makes little sense to include in your product feed any products that are not available to buy on your website.
The other initial setting which needs to be set is the Cron job reporting – all that is needed here is an email address that will receive reports on the success or otherwise of your scheduled data feeds. Whilst data feeds are a brilliant ‘set it up and leave it running’ tool, it’s vital that you have a means of being alerted if anything does go wrong.
Setting up Google Shopping
Because I did the majority of the work for Google Shopping, I’m going to use this as the main use case – the Google Shopping product feed is also probably the main reason why most Magento retailers tend to use Wyomind. The Google Shopping feed was originally supported by core Magento, but support for this was removed from Magento, when Google updated its Shopping API to v2 in early 2015. From that point on, it became necessary to switch to a third-party extension in order to feed into Google Shopping.
To feed data into Google Shopping, a retailer needs to have a Google Merchant Center account. For merchants who are not familiar with Google Shopping, it is wise to take time to read through Google’s own Merchant Center resources, to fully understand how the feed works, and what data you need to supply.
As this extension comes with a pre-built template to handle Google Shopping, there isn’t a huge amount of work to do in terms of configuration, aside from mapping the attributes and creating new ones in places. One key element that will need to be completed though, is category mapping. Since the categories in a Magento store are defined by the store owner, and are therefore not identical to the product categories used in Google Shopping, a mapping exercise is needed. The extension has a category mapping tool, which allows the store owner to view all Magento categories, and to find the closest-matching Google Shopping category using a smart auto-suggest tool.
Another configuration option that needs to be specified at this point is the frequency of feed creation. This is defined in the data feed tab labelled ‘Scheduled Task’. Feeds are generally created overnight, for example at 1am, in order to minimise any performance impact whilst the feed generation takes place. I set it to daily and the feed is sync’d at 4am.
Depending on what type of products are to be included in a feed, Google may require different attributes to be specified, in addition to the basic data fields such as name, description, price, availability, condition, GTIN etc. Obviously, if a field is required in order for the data feed to be valid, there is no choice but to include it. However, many fields are optional, but Wyomind can handle this, as it allows the user to include any Magento product attribute in any feed. By spending time at the setup stage, configuring the most comprehensive data feed possible, merchants will deliver the most data-rich feed into Google, and will hopefully reap the rewards going forward.
As well as the required fields, it is recommended that as many as possible additional fields are included in a Google Shopping feed, such as product type, additional images, sale prices (if appropriate), delivery costs, and Adwords attributes, which help with tracking Adwords campaigns.
Once the Google Shopping feed is configured in the Data Feed Manager, the feed has to be created and uploaded to Google Merchant Center. In the Merchant Center, click on Products, then Feeds. If no feed has been created already, click on the blue + symbol to create a new feed. Initially, this should be set as a test feed, to confirm all data is imported into Google Shopping correctly.
Whilst testing, it is fine to use the manual upload facility to load the data feed file into the Merchant Center, but ultimately, this will be handled via a scheduled upload, so that a freshly-generated data file is uploaded regularly, ideally daily.
Whilst normally, a product data feed should pass straight into the Merchant Center without errors, if there are any upload problems, these will be shown on the Diagnostics panel, to enable the store owner to rectify all issues.
Once the data feed has been tested thoroughly, the feed can be converted to a live feed, and scheduled for daily generation and upload. Click on ‘Create Schedule’ for your new feed in the Merchant Center, and specify what schedule you’d like to upload on. Ideally, you should set the Google upload schedule to be after the Magento feed creation schedule, to ensure that Google always retrieves the most up to date feed.
Adding new attributes
In order to get the feed approved, I needed to create a number of attributes that weren’t be used by the merchant previously. Some of these I was able to hard code in the data manager window, as can be seen for condition nad the identifier attributes below. This was a far quicker option than adding the attributes within Magento.
Other attributes that we needed to add were the GTIN, which we just added and then uploaded the data (as luckily the client already had it). We also needed to add a secondary gender attribute, to handle unisex as on the website they call it “men & women”, which isn’t supported by Google. We also needed to add some shipping data, some of which we hard coded, some we used the existing attributes and some we mapped within the merchant centre.
Filtering the products included in a data feed
There are a number of reasons why a merchant may need to restrict what products are supplied in a data feed. For example, we didn’t want the configurable products included in the feed and there were also a number of new products that we didn’t want to include for business reasons. Products that are available to pre-order only, or that have an extended delivery time are other examples of products that may not be suitable for inclusion in a data feed, as they could annoy the customer, who clicks to buy and then finds the product is not available immediately.
Fortunately, the Wyomind extension provides various options to allow the store owner to define exactly what products should be included in each data feed. I used the filter function to exclude everything that wasn’t a simple product, as can be seen below.
Filtering by category is the most basic form of product restriction, and this can be done on either an ‘include’ or an ‘exclude’ basis. Whether you choose to include or exclude specific categories will come down to what you want to achieve with the feed. Obviously, if you only want to include a single product category in the feed, using include would work well. If, however, you want to include your entire catalog, except for one or two categories, then using exclude would be the simplest option. You can see I removed two categories used for add-ons that you wouldn’t want to promote via Google Shopping.
Data Feed Manager contains further filters to enable the store owner to specify exactly what products are included. As I mentioned above, it is possible to filter based on product type, so that only simple products are included. We also used the option to filter attribute sets (which helped to remove custom options) and visibility (which we used to remove orphaned products that are pre-orderable), along with advanced filters that are based on any product attribute specified within the Magento store. For example, using advanced filters, a store owner could exclude products within a certain SKU range, or those from a specific manufacturer, if required. The advanced filters provide a huge degree of flexibility, when defining which products should be included in a feed. Within the general configuration options, it is possible to specify whether multiple filters should be treated with an AND operator, or an OR operator, for maximum control.
Data feeds for other shopping engines
The process for producing feeds for other shopping engines is broadly the same as for Google Shopping, although each engine may have slightly different upload formats and required fields. The Wyomind team has produced a comprehensive user guide though, to go with the 19 pre-installed data feed templates, to ensure merchants are clear on what is required for each data feed to be successful. PHP code snippets can even be included in feed templates, in order to format field contents to exactly what is required for any given shopping engine.
Ongoing support and upgrades
The initial purchase of this extension provides 3 months of support and upgrades. It is possible to extend this period at the checkout stage, for a small additional fee. Three months additional support will cost $30, whilst a full year of extra support will add $95 to the overall cost of the extension. This is money well-spent, as Wyomind has a good reputation when it comes to customer support, and the company offers a 24-hour guaranteed response on all support questions. It is also reassuring to know that the extension is regularly updated to cover any changes made by the shopping engines to their feed specifications.
I was pleasantly surprised at how simple this process was and I didn’t need any development resource at all. Whilst a degree of initial setup time is needed (for me this was mainly around mapping the attributes and creating additional data), data feeds generally require minimal ongoing maintenance.
With the Wyomind Data Feed Manager, the task is logical and straightforward, and merchants should be up and running in no time at all. This merchant was using Magento CE 1.9, however Wyomind also have both the Data Feed Manager and Simple Google Shopping extensions available for Magento 2.