Skip to content
oalders edited this page Mar 9, 2012 · 21 revisions

SysAdmin FAQ

Manual maintenance issues

How to reindex a missing module?

cd api.metacpan.org
bin/metacpan release http://cpan.metacpan.org/authors/id/X/XS/XSAWYERX/MetaCPAN-API-0.33.tar.gz --latest

How to index all unindexed dists uploaded over the last 24 hours?

bin/metacpan release --skip --age 24 --latest ~/CPAN/authors/id/

Restarting services

The following services are set up:

metacpan-web
metacpan-api
metacpan-rrr
metacpan-watcher
elasticsearch

Each of those services can be restarted by calling service $name restart (as superuser). Both metacpan-web and metacpan-api can also be restarted by the metacpan user using the rc command, e.g. rcmetacpan-web restart.

What is MySQL used for?

MetaCPAN uses Piwik for web analytics. It stores user logs and aggregated data in MySQL. The stats can be accessed via https://metacpan.org/piwik/.

How to deploy a new version of MetaCPAN-API and MetaCPAN-Web?

Log in as the metacpan user (which loads perlbrew automatically), go to the appropriate folder (~/metacpan.org or ~/api.metacpan.org), pull from github and restart the service (rcmetacpan-web restart or rcmetacpan-api restart).

How to increase storage space for ElasticSearch and the CPAN mirror?

The CPAN mirror and the ElasticSearch data are stored in /srv/cpan and /srv/elasticsearch, respectively. Those are filesystems on top of the LVM LVs /dev/mapper/metacpan-cpan and /dev/mapper/metacpan-elasticsearch.

To increase the space available on one of them, change the following example, which adds an additional 100 MB for the CPAN mirror. There's no need to unmount anything.

# Grow the LVM volume
lvextend -L +100M /dev/mapper/metacpan-cpan
# Extend the filesystem to the fit the new LV size
resize2fs /dev/mapper/metacpan-cpan

Currently there's 26.70 GiB unused space left to allocate for either ElasticSearch or CPAN.

Do NOT allocate all the unused space to logical volumes. We need some free space to use by LVM snapshots during the backup process. We haven't actually checked how much spare space we need for that, so let's play it safe and say that at least 1.5 GiB should be left alone.

Network/high level issues

Managing DNS

We are currently sponsored by Dyn with a DynECT Managed DNS Lite account.

URL: https://manage.dynect.net

When delegating your domain names, please use the following nameservers:

ns1.p24.dynect.net

ns2.p24.dynect.net

ns3.p24.dynect.net

ns4.p24.dynect.net

The best place to get started is look at the DynECT Managed DNS Lite User Manual located at: https://manage.dynect.net/help

Our contact at Dyn is Chris Gonyea [email protected]. Also, alh in #metacpan works for Dyn and can help with technical issues.

What is the procedure if the server is unreachable?

SpeedChilli

If you are a metacpan admin, you can contact support at speedchilli dot com - they will only respond to people who are on their list.

If out of hours, and it is an emergency that cannot wait, then contact emer-support instead (again, metacpan admins only). Be aware that people will be woken up to deal with the emergency, so please remain in contact to respond to questions.

Also, if we need to do anything (eg upgrade, reboot etc) which would set off alarms, please email support to let them know beforehand, and again when finished. Try to let them know during office hours if you are planning work out of office hours, and send an email 15-20 mins before starting work. Don't expect a response.

Booking.com

Contact the booking staff either by email ([email protected]) or in emergencies by phone (+31207153409). Most problems are better solved on IRC. Our contact on irc.perl.org is Seveas.

Where are system monitoring reports stored?

The ElasticSearch status can be queried from within the box:

$ curl localhost:9200/cpan/_status?pretty

http://munin.metacpan.org/org/metacpan.org/

http://nagios.omega.pqpq.de/

web api watcher

Where are backups stored

Clone this wiki locally