Skip to content


Mise en cache des métadonnées de yum avec nginx (avec docker) (pour Debian jessie)

Introduction

Le but de cette opération est de mettre en cache sur un serveur les métadonnées d'un dépôt particulier afin d'éviter les soucis tels que :

"14: PYCURL ERROR 6 - Couldn't resolve host"

En effet il arrive assez souvent que la réponse envoyant les métadonnées se fasse attendre !

Une solution est de mettre en cache ces métadonnées sur un serveur faisant office de mandataire.

Mise en place

Installer nginx :

apt-get install nginx -y

Tout en vrac......

cat /etc/nginx/nginx.conf

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
events {
worker_connections 1024;
}
http {
log_format main '$remote_addr - cache: $upstream_cache_status - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
 
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
include /etc/nginx/conf.d/*.conf;
}

 

cat /etc/nginx/conf.d/epel.conf

proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=STATIC:50m inactive=200d max_size=12g;
server {
listen 81;
server_name epel.local;
 
location /pub/ {
proxy_pass https://dl.fedoraproject.org/pub/;
proxy_cache off;
proxy_no_cache 1;
expires off;
}
 
# cache repodata
location /pub/epel/6/x86_64/repodata/ {
proxy_pass https://dl.fedoraproject.org/pub/epel/6/x86_64/repodata/;
include /etc/nginx/cache.conf;
}
 
location /icons/ {
proxy_pass https://dl.fedoraproject.org/icons/;
include /etc/nginx/cache.conf;
}
}

 

cat /etc/nginx/cache.conf

proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size 100m;
client_body_buffer_size 1m;
proxy_connect_timeout 900;
proxy_send_timeout 900;
proxy_read_timeout 900;
proxy_buffers 32 4k;
proxy_cache STATIC;
proxy_cache_valid 200 10m;
proxy_ignore_headers X-Accel-Expires Expires Cache-Control;

 

Le cas du repo EPEL

cat /etc/yum.repos.d/epel.repo

[epel-cache]
name=Extra Packages for Enterprise Linux 6 - $basearch
baseurl=http://IP-container:8000/pub/epel/6/$basearch
#baseurl=https://dl.fedoraproject.org/pub/epel/6/x86_64/
#mirrorlist=http://centos.mirrors.ovh.net/ftp.centos.org/6/os/x86_64/
failovermethod=priority
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6

On lance tout cela :

docker rm -f some-nginx ; docker run -it --name some-nginx -p 8000:81 -v `pwd`/nginx.conf:/etc/nginx/nginx.conf:ro -v `pwd`/cache.conf:/etc/nginx/cache.conf:ro -v `pwd`/conf.d/epel.conf:/etc/nginx/conf.d/epel.conf:ro -d nginx

Puis on log :

clear ; docker logs -f some-nginx
 
10.0.2.38 - cache: EXPIRED - - [03/Aug/2016:14:48:55 +0000] "GET /pub/epel/6/x86_64/repodata/repomd.xml HTTP/1.1" 200 4380 "-" "urlgrabber/3.9.1 yum/3.2.29" "-"
10.0.2.38 - cache: EXPIRED - - [03/Aug/2016:14:49:45 +0000] "GET /pub/epel/6/x86_64/repodata/b62b82980a3683d984a9ce7444172b512d356ce33202717ad4c20b19e21c2db0-primary.sqlite.bz2 HTTP/1.1" 200 6135536 "-" "urlgrabber/3.9.1 yum/3.2.29" "-"
10.0.2.38 - cache: - - - [03/Aug/2016:14:50:58 +0000] "GET /pub/epel/6/x86_64/htop-1.0.3-1.el6.x86_64.rpm HTTP/1.1" 200 85176 "-" "urlgrabber/3.9.1 yum/3.2.29" "-"
10.0.2.38 - cache: - - - [03/Aug/2016:14:51:09 +0000] "GET /pub/epel/6/x86_64/libwebp-0.4.3-3.el6.x86_64.rpm HTTP/1.1" 200 215532 "-" "urlgrabber/3.9.1 yum/3.2.29" "-"
10.0.2.38 - cache: - - - [03/Aug/2016:14:51:13 +0000] "GET /pub/epel/6/x86_64/munin-2.0.25-11.el6.noarch.rpm HTTP/1.1" 200 203772 "-" "urlgrabber/3.9.1 yum/3.2.29" "-"
10.0.2.38 - cache: - - - [03/Aug/2016:14:51:14 +0000] "GET /pub/epel/6/x86_64/munin-common-2.0.25-11.el6.noarch.rpm HTTP/1.1" 200 93556 "-" "urlgrabber/3.9.1 yum/3.2.29" "-"
10.0.2.38 - cache: - - - [03/Aug/2016:14:51:21 +0000] "GET /pub/epel/6/x86_64/munin-node-2.0.25-11.el6.noarch.rpm HTTP/1.1" 200 413312 "-" "urlgrabber/3.9.1 yum/3.2.29" "-"
10.0.2.38 - cache: - - - [03/Aug/2016:14:51:26 +0000] "GET /pub/epel/6/x86_64/perl-Net-CIDR-0.18-1.el6.noarch.rpm HTTP/1.1" 200 19304 "-" "urlgrabber/3.9.1 yum/3.2.29" "-"
10.0.2.38 - cache: - - - [03/Aug/2016:14:51:47 +0000] "GET /pub/epel/6/x86_64/python2-boto-2.40.0-1.el6.noarch.rpm HTTP/1.1" 200 1872764 "-" "urlgrabber/3.9.1 yum/3.2.29" "-"
10.0.2.38 - cache: - - - [03/Aug/2016:14:51:48 +0000] "GET /pub/epel/6/x86_64/python2-rsa-3.4.1-1.el6.noarch.rpm HTTP/1.1" 200 69708 "-" "urlgrabber/3.9.1 yum/3.2.29" "-"

😉

Print Friendly

Posted in CentOS, Debian, Linux, Technique, Toutes.

Tagged with , , , , , , .


No Responses (yet)

Stay in touch with the conversation, subscribe to the RSS feed for comments on this post.



Some HTML is OK

or, reply to this post via trackback.

Time limit is exhausted. Please reload CAPTCHA.