Catalogue of industrial land and business centres in the province of Seville

This time, we present a project to modernize the catalog of industrial land and business centers that Prodetur, a subsidiary of the Seville Provincial Council, has been maintaining for years.

The main challenges of this project were: migrating the data from the previous platform, integrating it with the Google Maps platform, and making the necessary adjustments to ensure that the latter efficiently displayed the nearly 30,000 spaces and business centers.

Migration from Drupal 7

The previous system had been developed in Drupal 7, so one of the most important tasks was maintaining the cataloged data in the new system.

For this migration, we used both the migration modules included in the Drupal core and others contributed by the community, as well as small custom-developed add-ons.

Drupal's migration system allows you to define migrations as text files that describe, for each one: its origin, how its data is processed, and its destination. These migrations, once defined, can be executed (fully or partially). This allows you to verify that the content has been migrated correctly. It also allows you to revert the imports, deleting all the created content. This reduces feedback cycles and speeds up what is otherwise a long and tedious procedure.

Furthermore, the system is extensible using Drupal plugins, allowing each migration to be adapted to the specific needs of each project. For example, in this project, each company (of which nearly 9,000 were migrated) defines its activity with a CNAE code (a standard classification of economic activities in Spain). CNAE codes are up to eight digits long and are organized in a tree structure. Thus, activities with more digits define the activity performed more precisely. To improve the site's usability, it was decided to reduce the number of digits in the stored codes, thereby simplifying searches. This was achieved with a small plugin that processed the CNAE values ​​during the migration.

Integration with Google Maps

The core of the platform consists of maps that display the location of each cataloged item.

After several prototypes, an integration based on contributed modules was chosen for content editing, along with a headless solution for the public-facing part of the site.

The contributed module used, Geofield, extends the system by adding a field type for storing coordinates (both single and multiple coordinates in GeoJSON or KML format) and various form elements, such as a search bar that integrates with Google Places and a map where users can click on the exact location.

The public-facing part of the site faced the challenge of displaying nearly 30,000 items simultaneously. Therefore, a headless solution was chosen, where Drupal generates web services that the user's browser's JavaScript code then efficiently downloads, displays, and filters, as explained in the next section.

Performance Tuning

Displaying nearly 30,000 items on a single page is quite a challenge. Drupal can efficiently generate web services and has an excellent caching layer for this purpose, but 30,000 items is still too many. Conventional listings, where items are displayed one after another, use pagination, but this option isn't viable for a map.

The first decision was to separate the map generation from the generation of the items it displays. This allowed us to create a static page that users download instantly and that handles downloading the information for each item to be displayed.

Also, to improve the download time of these items, the Search API module was used to generate an index with the information ready for consumption. When generating listings, Drupal loads each item into memory, executing its entire loading cycle at that moment and increasing the listing generation time to several minutes. The Search API, on the other hand, allows you to declare which data to use and prepares it as new content is inserted or updated, making list generation much faster.

On the browser side, some optimizations were also made to the JavaScript code.

Taking advantage of the pagination available in Drupal listings, the data is downloaded in sets of 2,500 items. All requests are made in parallel, allowing the browser to manage them in its download queues. Depending on the browser, up to six of these pages have been observed to be downloaded in parallel.

Point clustering was implemented. This functionality automatically groups points according to their distance, creating markers that zoom in on points when clicked.

Only visible points are added to the map at any given time. Each time the user moves the map, scrolling or zooming, the visible points are reviewed, adding or removing markers as needed. This improves the map's responsiveness and makes it more user-friendly.