Integrating MediaWiki with External Databases and APIs for Seamless Data Exchange
MediaWiki is a flexible platform capable of integrating with external systems to enrich content, automate updates, and provide dynamic data displays. By connecting to APIs, databases, and other services, you can create hybrid wiki applications that combine the collaborative power of MediaWiki with real-time data from other systems. This tutorial provides a technical walkthrough of best practices for integrating MediaWiki with external data sources, covering database connections, REST API consumption, caching strategies, security considerations, and implementation patterns.
To begin, determine the nature of the external data you want to include in your wiki. Common examples include product inventory, user directories, scientific datasets, or metrics from monitoring systems. Knowing the data structure, update frequency, and required presentation format helps guide integration methods.
One integration approach is direct database access. MediaWiki’s database abstraction layer allows secure connectivity to external MySQL or PostgreSQL databases. To connect to a remote database, start by adding connection credentials to your LocalSettings.php. Define a new load balancer entry, for example:
$wgExternalDatabases['myexternal'] = [ 'class' => 'DatabaseLoadBalancer', 'hosts' => [ [ 'host' => 'external-db.example.com', 'dbname' => 'externaldb', 'user' => 'wikiuser', 'password' => 'secret', ] ] ];
After configuring the connection, you can use MediaWikiServices to obtain a database object:
$db = MediaWiki\\MediaWikiServices::getInstance()->getDatabaseLoadBalancer()->getConnectionRef( DB_REPLICA, 'myexternal' );
Use $db->select, $db->insert, or $db->update to work with tables in the external database. It is critical to follow best practices for prepared statements and to avoid SQL injection.
For more advanced use cases, consider using the External Data extension. It simplifies accessing remote database tables or APIs and importing values into wiki pages. For example, using parser functions like #get_db or #get_web_data, you can fetch values in wikitext syntax. This is ideal for light data integration without custom coding.
If the external data is accessible via REST APIs, writing a custom parser function or hook is often the best approach. Create a parser function by registering it in extension.json and implementing the handler in PHP. Use PHP built-in HTTP clients like HttpRequestFactory or cURL to fetch data. For example, to retrieve JSON from https://api.example.com/data/123, write:
$response = HttpRequestFactory::create( 'https://api.example.com/data/123' )->execute();
Parse the response using FormatJson::decode( $response->getContent() ); and extract key fields. Make sure to handle HTTP errors, timeouts, and network failures gracefully.
Caching API responses is essential. Avoid running an external request on every page view. Use the MediaWiki job queue or parser cache. For immediate caching, store results in MediaWiki’s object cache using a key derived from the request URL. For example:
$key = 'externalapi:' . sha1( $url );
$cache = MediaWikiServices::getInstance()->getMainWANObjectCache();
$data = $cache->get( $key );
if ( $data === false ) { $data = // execute HTTP request and decode; $cache->set( $key, $data, 3600 ); }
Get Hex Shift’s stories in your inbox
Join Medium for free to get updates from this writer.
This caches the API response for one hour.
For larger datasets, use asynchronous job queuing. Define a job in extension.json and register a job handler class. This job fetches data from the API and stores it in a local wiki table. On page render, query this table for display. Running jobs periodically keeps the local cache updated and prevents user-experienced delays.
Security is critical. Never expose API keys or credentials in public wiki pages. Store secrets in environment variables or use secrets management services. For database integrations, ensure that user accounts have limited privileges such as read-only access to necessary tables. Use TLS for all external connections.
If the API provides authentication such as OAuth or token-based access, implement a secure flow. Use server-side token storage and refresh logic. Avoid storing tokens in the database without encryption.
Integrating MediaWiki with modern services often requires parsing HTML or XML. Use Html::parseFragment and Html::element helpers to sanitize and generate content. Avoid string concatenation to prevent XSS. If transforming large HTML structures or SVG, strip dangerous attributes before rendering.
When integrating data, consider how to format it visually. Use HTML tables, charts via Chart.js or Google Charts, or interactive maps. Register ResourceLoader modules to load JavaScript libraries and attach them conditionally. For example:
$out->addModules( 'ext.myapi.charts' );
This allows you to render data in graphs rather than raw tables.
Integration with external data should also consider fallback scenarios. If API is offline, present stale data or a friendly message. Use try-catch blocks in PHP and treat user experience carefully.
For power users, Lua modules offer a performant way to process external data. Write a Lua script to query a local cache table and generate table rows dynamically. Use the Scribunto extension to execute Lua at render time with minimal overhead.
Testing integrated functionality is vital. Write PHPUnit test cases for parser functions and hook handlers. Mock HTTP responses using tools like MockHttpClient. For job handling and database updates, write tests using DatabaseTestCase in PHPUnit.
Version control and deployment should treat external integrations with care. Store credentials outside version control. Use deployment scripts to inject environment variables or secrets on each host. Maintain separate API endpoints for staging and production.
Keep monitoring logs and metrics related to integration. Log external failures and increase visibility on slow API responses. Use web analytics or Grafana dashboards to track integration usage and latency.
Integration can enable powerful use cases. Public wikis can display live stock prices, weather, or system status. Internal wikis can show ticket counts, CI pipeline statuses, or inventory data. Learning to build secure, reliable integrations expands the possibilities of MediaWiki far beyond static documents.
In conclusion, integrating MediaWiki with external databases and APIs enhances the utility and relevance of your wiki by providing dynamic content that updates automatically. This approach requires careful handling of connections, caching, security, presentation, and error handling. By following best practices outlined in this article, you can create integrations that are both powerful and stable.
For more advanced techniques including configuration snippets, job definition examples, secure token storage, and high performance query optimization, consult Mastering MediaWiki Extensions: Beyond the Manual. This guide provides expert methods and real world examples to help you build robust MediaWiki integrations at scale.
