How to mirror CPAN
There are several ways to mirror CPAN depending upon what you want to achieve.
How do I create a private or offline mirror?
Requirements for a full / public mirror
- Good internet connectivity
- Around 1GB of storage space for just the current modules.
- Around 11GB of storage space for the full mirror.
It's highly recommended that you also subscribe to the announcements-only cpan-mirrors mailing list by emailing cpan-mirrors-subscribe at perl.org.
Only use FTP if these other methods are absolutely impossible. Never mirror with HTTP - you will end up with a million duplicate files in tens of gigabytes.
Which CPAN Mirror should I use?
Near future: Tier 1 mirrors are instantly updated and can be accessed as
Current: Find your nearest rsync enabled site on http://www.cpan.org/SITES.html, or use mirrors.json especially if you are building a tool which lets the user select a mirror. You can also sync from rsync://cpan.rsync.perl.org/CPAN/, though you currently might get better performance from a "local" mirror.
Please limit to once or twice a day. For more frequent updates please see Instant mirroring.
On Unix systems
/usr/bin/rsync -av --delete cpan-rsync.perl.org::CPAN /project/CPAN/
Using 'crontab' you can make rsync run once a day, for example
4 40 * * * sleep $(expr $RANDOM \% 7200); /usr/bin/rsync -av --delete cpan-rsync.perl.org::CPAN /project/CPAN/
The "sleep $(...);" statement makes the command delay up to 2 hours before running rsync; the advantage of this is that you (and everybody else) won't access the mirror at the same time!
On Windows systems
C:\Program Files\Rsync\rsync -av --delete cpan-rsync.perl.org::CPAN /project/CPAN/
Using the 'AT' tool, you can schedule rsync to run daily, for example:
AT 20:00 /every:M,T,W,Th,F,S,Su "C:\Program Files\Rsync\rsync -av --delete cpan-rsync.perl.org::CPAN /project/CPAN/"
How do I create a public mirror?
- Consider Instant mirroring, required if you wish to be a tier 1 mirror, or..
- rsync once a day
- Provide (in order of preference) rsync, HTTP and/or FTP public access
- To be added to http://www.cpan.org/SITES.html and mirrors.json please complete the template confirming the public accessible URLs to your mirror: rsync, ftp, http and email it to firstname.lastname@example.org.
Please join the trial. This will become the preferred method of mirroring for public sites, it is required for all Tier 1 mirrors. Currently being tested.