目录

一、仓库类型

二、Harbor组件说明

三、harbor单机版部署

1、添加主机名解析

2、安装docker

3、安装docker-compose

4、安装harbor

5、配置CA为harbor签发证书/生成V3证书

1) 创建CA证书

2) 为harbor服务器创建证书

3）创建v3.ext文件

6、编辑harbor配置文件

7、启动harbor

四、访问harbor

1、上传镜像

1) 登录仓库

2) 上传镜像

​五、部署Harbor高可用集群

1) 环境描述

2) 在两台Harbor服务器分别安装docker、docker-compose、harbor

3) 配置NFS服务器

3) 配置harbor服务挂载NFS存储

4) 部署redis服务

5) 部署postgreSQL

a. 安装postgreSQL

b. 初始化postgreSQL

c. 编辑postgreSQL配置文件

d. 指定允许远程连接数据库的客户端

e. 启动数据库

f. 创建harbor需要的数据库及远程连接用户

g. 远程主机测试连接数据库

6、安装harbor

7、编辑harbor配置文件

8、启动harbor

9、测试harbor集群

        Harbor（Harbor Container Registry）是一个开源的企业级容器镜像注册中心（Container Registry），旨在提供安全、可靠、高度可扩展的容器镜像管理和分发解决方案。Harbor支持Docker镜像，并提供了多种功能，适用于容器化应用的构建、部署和管理。

 看到这里有小伙伴不明白有docker的，可以去看看这篇文章，对小白非常的友好，内容非常详细。容器化奇迹：打开创新之门，加速业务发展的秘密武器

        在构建镜像仓库之前，我们先了解一下仓库的类型

一、仓库类型

市面上还有很多很多仓库类型，今天我们就用开源的最经典的harbor仓库，为大家讲讲如何构建以及如何使用harbor仓库

二、Harbor组件说明

在使用之前呢，我们先来了解一下harbor的原理，以及各个组件之间的关系

  • redis
    • 负责存储会话信息
  • harbor-db
    • 负责存储仓库数据(项目名称、镜像名称)
  • registry
    • 镜像的上传、下载
  • Log collector
    • 记录日志
  • harbor-core
    • 提供web UI界面
    • 颁发令牌token
    • webhook
      • 负责搜集镜像变化的信息，通知给UI进行展示
  • proxy
    • 提供反向代理
  • Job service
    • 提供镜像复制功能

三、harbor单机版部署

1、添加主机名解析
[root@harbor ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.140.11	harbor.linux.com
2、安装docker
[root@harbor ~]# rpm -q docker-ce
docker-ce-20.10.7-3.el7.x86_64

修改镜像下载路径
[root@harbor ~]# cat /etc/docker/daemon.json 
{
    "registry-mirrors": ["http://f1361db2.m.daocloud.io"]
}
3、安装docker-compose
这里已经提前下好了docker-compose，直接移动到bin目录下生效
[root@harbor ~]# mv docker-compose /usr/local/bin/
[root@harbor ~]# chmod a+x /usr/local/bin/docker-compose
4、安装harbor

harbor的下载地址：https://github.com/goharbor/harbor/tags

这里也已经提前下载好了harbor压缩包
[root@harbor ~]# tar xf harbor-offline-installer-v2.2.2.tgz 
[root@harbor ~]# cp harbor/harbor.yml.tmpl harbor/harbor.yml
5、配置CA为harbor签发证书/生成V3证书
1) 创建CA证书
mkdir /opt/ssl
cd /opt/ssl
openssl genrsa -out ca.key 4096
ls
openssl req -x509 -new -nodes -sha512 -days 3650  -subj "/CN=harbor.linux.com"  -key ca.key  -out ca.crt
2) 为harbor服务器创建证书
openssl genrsa -out server.key 4096

openssl req  -new -sha512  -subj "/CN=harbor.linux.com"  -key server.key  -out server.csr
3）创建v3.ext文件
[root@localhost ssl]# cat v3.ext 
 authorityKeyIdentifier=keyid,issuer
 basicConstraints=CA:FALSE
 keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
 extendedKeyUsage = serverAuth 
 subjectAltName = @alt_names
 [alt_names]
 DNS.1=harbor.linux.com

[root@localhost ssl]# openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key -CAcreateserial -in server.csr -out server.crt
6、编辑harbor配置文件
[root@harbor harbor]# vim harbor.yml

hostname: harbor.linux.com

# http related config
http:
  # port for http, default is 80. If https enabled, this port will redirect to https port
  port: 80

# https related config
https:
  # https port for harbor, default is 443
  port: 443
  # The path of cert and key files for nginx
  certificate: /opt/ssl/server.crt
  private_key: /opt/ssl/server.key
7、启动harbor
[root@harbor harbor]# ./prepare 
prepare base dir is set to /root/harbor
Unable to find image 'goharbor/prepare:v2.2.2' locally
v2.2.2: Pulling from goharbor/prepare
b31150c04016: Pull complete 
d504272addf9: Pull complete 
a9c2d9be0ec7: Pull complete 
ba14108b237f: Pull complete 
888a2dd12a77: Pull complete 
08591f736052: Pull complete 
e9a06c50605c: Pull complete 
fcc257111f80: Pull complete 
Digest: sha256:d12185f2c925416fa260d2af8764d8c27d35b4f66d9bcff67bf7e35d9409789e
Status: Downloaded newer image for goharbor/prepare:v2.2.2
Generated configuration file: /config/portal/nginx.conf
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/registryctl/config.yml
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Generated and saved secret to file: /data/secret/keys/secretkey
Successfully called func: create_root_cert
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir
[root@harbor harbor]# ./install.sh 

Creating network "harbor_harbor" with the default driver
Creating harbor-log ... done
Creating harbor-db     ... done
Creating harbor-portal ... done
Creating redis         ... done
Creating registryctl   ... done
Creating registry      ... done
Creating harbor-core   ... done
Creating nginx             ... done
Creating harbor-jobservice ... done
✔ ----Harbor has been installed and started successfully.----

[root@harbor harbor]# docker image ls
REPOSITORY                      TAG       IMAGE ID       CREATED        SIZE
goharbor/harbor-exporter        v2.2.2    d63334489be4   2 months ago   76.4MB
goharbor/chartmuseum-photon     v2.2.2    c3c6b2f81c7c   2 months ago   165MB
goharbor/redis-photon           v2.2.2    11a777ead643   2 months ago   69MB
goharbor/trivy-adapter-photon   v2.2.2    a0cd0b967c47   2 months ago   120MB
goharbor/notary-server-photon   v2.2.2    f963e34d9130   2 months ago   102MB
goharbor/notary-signer-photon   v2.2.2    c3ce4df1b104   2 months ago   98.5MB
goharbor/harbor-registryctl     v2.2.2    461e67c4ec3d   2 months ago   128MB
goharbor/registry-photon        v2.2.2    fb9295e771e6   2 months ago   77.3MB
goharbor/nginx-photon           v2.2.2    6744b15891f6   2 months ago   40.4MB
goharbor/harbor-log             v2.2.2    3837bbad1328   2 months ago   108MB
goharbor/harbor-jobservice      v2.2.2    c9db96b1363f   2 months ago   163MB
goharbor/harbor-core            v2.2.2    2e6b040afa40   2 months ago   148MB
goharbor/harbor-portal          v2.2.2    c240f7374709   2 months ago   51.1MB
goharbor/harbor-db              v2.2.2    e6b67be16b5b   2 months ago   177MB
goharbor/prepare                v2.2.2    eb938b7f85aa   2 months ago   165MB
[root@harbor harbor]# docker ps -a
CONTAINER ID   IMAGE                                COMMAND                  CREATED         STATUS                        PORTS                                                                            NAMES
2c20abb4d255   goharbor/harbor-jobservice:v2.2.2    "/harbor/entrypoint.…"   2 minutes ago   Up About a minute (healthy)                                                                                    harbor-jobservice
b5393b959230   goharbor/nginx-photon:v2.2.2         "nginx -g 'daemon of…"   2 minutes ago   Up About a minute (healthy)   0.0.0.0:80->8080/tcp, :::80->8080/tcp, 0.0.0.0:443->8443/tcp, :::443->8443/tcp   nginx
64f26bafbd90   goharbor/harbor-core:v2.2.2          "/harbor/entrypoint.…"   2 minutes ago   Up 2 minutes (healthy)                                                                                         harbor-core
6c8a836e6c10   goharbor/registry-photon:v2.2.2      "/home/harbor/entryp…"   2 minutes ago   Up 2 minutes (healthy)                                                                                         registry
048bb45ebb42   goharbor/redis-photon:v2.2.2         "redis-server /etc/r…"   2 minutes ago   Up 2 minutes (healthy)                                                                                         redis
bf345e4dbce7   goharbor/harbor-registryctl:v2.2.2   "/home/harbor/start.…"   2 minutes ago   Up 2 minutes (healthy)                                                                                         registryctl
26eaff3ead77   goharbor/harbor-portal:v2.2.2        "nginx -g 'daemon of…"   2 minutes ago   Up 2 minutes (healthy)                                                                                         harbor-portal
b57fdf68b23a   goharbor/harbor-db:v2.2.2            "/docker-entrypoint.…"   2 minutes ago   Up 2 minutes (healthy)                                                                                         harbor-db
61261e5bb47c   goharbor/harbor-log:v2.2.2           "/bin/sh -c /usr/loc…"   2 minutes ago   Up 2 minutes (healthy)        127.0.0.1:1514->10514/tcp                                                        harbor-log

四、访问harbor

https://harbor.linux.com

  • 创建项目
  • 创建用户
  • 用户授权
1、上传镜像
1) 登录仓库
Harbor仓库启用了https后，docker服务器要能正常登录访问仓库，需要将harbor的证书拷贝到docker服务器

[root@test ~]# mkdir /etc/docker/certs.d/harbor.linux.com -p

[root@harbor ~]# scp -a /opt/ssl/harbor.crt root@192.168.140.10:/etc/docker/certs.d/harbor.linux.com
[root@localhost ~]# docker login harbor.linux.com
Username: sam
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
2) 上传镜像
[root@localhost ~]# docker tag nginx:1.14 harbor.linux.com/project_yw/nginx:1.14 
[root@localhost ~]# docker push harbor.linux.com/project_yw/nginx:1.14

3)查看上传的镜像


五、部署Harbor高可用集群

1、高可用方案介绍

  • 利用Harbor自带的复制功能
    • 两个harbor节点在进行镜像复制时，如果其中一个节点故障, 此时有新镜像的话，并不会复制到故障节点，而且故障的harbor修复后也不会自动复制数据，会导致数据不一致问题
  • 基于共享存储的方式实现
    • 将数据库、redis独立出来
      • harbor 1.6版本后，数据库仅支持postgreSQL
    • 利用共享存储(nfs, ipsan, glusterfs)存储镜像

2、部署Harbor高可用集群

1) 环境描述

192.168.140.11 harbor服务器
192.168.140.13 harbor服务器
192.168.140.12 NFS、Redis、PostgreSQL

2) 在两台Harbor服务器分别安装docker、docker-compose、harbor

3) 配置NFS服务器
[root@share_storage ~]# mkdir /data/harbor -p
[root@share_storage ~]# chmod o+w /data/harbor/
[root@share_storage ~]# yum install -y nfs-utils rpcbind 
[root@share_storage ~]# vim /etc/exports
[root@share_storage ~]# cat /etc/exports
/data/harbor	192.168.140.11(rw,no_root_squash)   192.168.140.13(rw,no_root_squash)
 
[root@share_storage ~]# systemctl start nfs-server
[root@share_storage ~]# systemctl enable nfs-server.service 
Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service.

[root@share_storage ~]# showmount -e localhost 
Export list for localhost:
/data/harbor 192.168.140.13,192.168.140.11
3) 配置harbor服务挂载NFS存储
[root@harbor_01 ~]# yum install -y nfs-utils 

[root@harbor_01 ~]# vim /etc/fstab 
192.168.140.12:/harbordata	/data	nfs	defaults 0 0

[root@harbor_01 ~]# df -hT | grep nfs
192.168.140.12:/harbordata nfs4       18G  3.0G   15G  18% /data
4) 部署redis服务
[root@share_storage ~]# yum install -y redis
[root@share_storage ~]# vim /etc/redis.conf 
bind 192.168.140.12
daemonize yes
[root@share_storage ~]# systemctl start redis
[root@share_storage ~]# systemctl enable redis
[root@share_storage ~]# netstat -antp | grep redis
tcp        0      0 192.168.140.12:6379     0.0.0.0:*               LISTEN      17493/redis-server
5) 部署postgreSQL
a. 安装postgreSQL
[root@storage ~]# yum install -y cmake gcc gcc-c++ perl readline readline-devel openssl openssl-devel zlib zlib-devel ncurses-devel readline readline-devel zlib zlib-devel
[root@storage ~]# wget https://ftp.postgresql.org/pub/source/v12.2/postgresql-12.2.tar.gz
[root@storage ~]# tar xf postgresql-12.2.tar.gz 
[root@storage ~]# cd postgresql-12.2/
[root@storage ~]#./configure --prefix=/usr/local/postsql
[root@storage ~]# make && make install
b. 初始化postgreSQL
[root@storage ~]# useradd postgres
[root@storage ~]# mkdir -p /work/harbor-db/{data,temp,log}
[root@storage ~]# chown -R postgres.postgres /work/harbor-db
[root@storage ~]# /usr/local/postgresql/bin/initdb --username=postgres -D /work/harbor-db/data/
c. 编辑postgreSQL配置文件
[root@storage ~]# vim /work/harbor-db/data/postgresql.conf 

data_directory = '/work/harbor-db/data'

listen_addresses = '*'          
port = 5432                           
max_connections = 100   

unix_socket_directories = '/work/harbor-db/temp'
unix_socket_group = ''
unix_socket_permissions = 0777

shared_buffers = 128MB

timezone = 'Asia/Shanghai'
 
logging_collector = on
log_directory = '/work/harbor-db/log'
log_rotation_size = 1GB
log_timezone = 'Asia/Shanghai'
log_min_duration_statement = 100
d. 指定允许远程连接数据库的客户端
[root@storage ~]# tail -n 2 /work/harbor-db/data/pg_hba.conf 
host    all		harbor          192.168.140.11/24                 trust
host    all		harbor		192.168.140.13/24                 trust
e. 启动数据库
[root@storage ~]# su - postgres
[root@storage ~]# /usr/local/postgresql/bin/pg_ctl -D /work/harbor-db/data/ -l /work/harbor-db/log/start.log start

[postgres@storage ~]$ netstat -antp | grep 5432
tcp        0      0 0.0.0.0:5432            0.0.0.0:*               LISTEN      32215/postgres
f. 创建harbor需要的数据库及远程连接用户
[root@storage ~]# /usr/local/postgresql/bin/psql -h 127.0.0.1 -p 5432 -U postgres

postgres=# \password postgres
Enter new password: 
Enter it again: 

postgres=# create user harbor with password 'redhat';
CREATE ROLE

postgres=# CREATE DATABASE harbor;
CREATE DATABASE

postgres=# create database harbor_clair;
CREATE DATABASE

postgres=# create database harbor_notary_server;
CREATE DATABASE

postgres=# create database harbor_notary_signer; 
CREATE DATABASE
harbor		
postgres=# GRANT ALL ON DATABASE harbor to harbor;           
GRANT

postgres=# GRANT ALL ON DATABASE harbor_clair to harbor;           
GRANT

postgres=# GRANT ALL ON DATABASE harbor_notary_server to harbor;            
GRANT

postgres=# GRANT ALL ON DATABASE harbor_notary_signer to harbor;                    
GRANT
g. 远程主机测试连接数据库
[root@harbor_02 ~]# psql -h 192.168.140.12 -p 5432 -U harbor -W
6、安装harbor
[root@harbor_01 ~]# tar xf harbor-offline-installer-v2.2.1.tgz
7、编辑harbor配置文件
[root@harbor_02 ~]# vim harbor/harbor.yml

hostname: harbor.linux.com

注释https相关配置，本案例中没有配置证书
#https:
	# port: 443
	# certificate: /usr/local/harbor/ssl/harbor.ssl	
	# private_key: /usr/local/harbor/ssl/harbor.key

data_volume: /data

harbor_admin_password: Harbor12345


注释或删除harbor自带的数据库配置，后续配置外部数据库连接
#database:
	# password: root123
	# max_idle_conns: 50
	# max_open_conns: 1000

配置harbor连接外部postgresql数据库
external_database:
  harbor:
    host: 192.168.140.12
    port: 5432
    db_name: harbor
    username: harbor
    password: redhat
    ssl_mode: disable
    max_idle_conns: 2
    max_open_conns: 0
  notary_signer:
    host: 192.168.140.12
    port: 5432
    db_name: harbor_notary_signer
    username: harbor
    password: redhat
    ssl_mode: disable
  notary_server:
    host: 192.168.140.12
    port: 5432
    db_name: harbor_notary_server
    username: harbor
    password: redhat
    ssl_mode: disable

配置连接外部redis存储会话信息
external_redis:
   host: 192.168.140.12:6379
   password:
   
   registry_db_index: 1
   jobservice_db_index: 2
   chartmuseum_db_index: 3
   trivy_db_index: 5
   idle_timeout_seconds: 30
8、启动harbor
[root@harbor_01 harbor]# ./prepare
[root@harbor_01 harbor]# ./install.sh 

[root@harbor_02 harbor]# docker ps -a
CONTAINER ID   IMAGE                                COMMAND                  CREATED          STATUS                             PORTS                                                                            NAMES
7839506e6909   goharbor/nginx-photon:v2.2.2         "nginx -g 'daemon of…"   19 seconds ago   Up 16 seconds (health: starting)   0.0.0.0:80->8080/tcp, :::80->8080/tcp, 0.0.0.0:443->8443/tcp, :::443->8443/tcp   nginx
0685bef08838   goharbor/harbor-jobservice:v2.2.2    "/harbor/entrypoint.…"   19 seconds ago   Up 16 seconds (health: starting)                                                                                    harbor-jobservice
0745cdc06650   goharbor/harbor-core:v2.2.2          "/harbor/entrypoint.…"   19 seconds ago   Up 18 seconds (health: starting)                                                                                    harbor-core
2dca99a9b8d9   goharbor/harbor-portal:v2.2.2        "nginx -g 'daemon of…"   20 seconds ago   Up 19 seconds (health: starting)                                                                                    harbor-portal
3b3f3a4d4192   goharbor/registry-photon:v2.2.2      "/home/harbor/entryp…"   20 seconds ago   Up 19 seconds (health: starting)                                                                                    registry
db630535b7e8   goharbor/harbor-registryctl:v2.2.2   "/home/harbor/start.…"   20 seconds ago   Up 19 seconds (health: starting)                                                                                    registryctl
27103477a551   goharbor/harbor-log:v2.2.2           "/bin/sh -c /usr/loc…"   21 seconds ago   Up 20 seconds (health: starting)   127.0.0.1:1514->10514/tcp

注意：所有Harbor节点配置保持一致

9、测试harbor集群

连接任意一台harbor上传镜像，上传镜像完毕后。修改本地hosts文件解析到其他harbor服务器，再次访问harbor web界面。第一会发现不需要登录直接可以访问数据，第二会看到同样的项目及镜像

# 容器 # 运维 # docker
Logo
开放原子开发者工作坊

开放原子开发者工作坊旨在鼓励更多人参与开源活动，与志同道合的开发者们相互交流开发经验、分享开发心得、获取前沿技术趋势。工作坊有多种形式的开发者活动，如meetup、训练营等，主打技术交流，干货满满，真诚地邀请各位开发者共同参与！

更多推荐

以智能 致世界 | 操作系统大会2024议程全览

avatar 开放原子开发者工作坊

开放原子开源基金会新增捐赠人（2024年9月）

2024年9月，新增以下单位成为开放原子开源基金会及旗下项目捐赠人。

avatar 开放原子开发者工作坊

2024开放原子开发者大会议题征集正式开启

2024开放原子开发者大会（以下简称“大会”）将于12月中旬重磅来袭,2024开放原子开发者大会议题征集正式开启!

avatar 开放原子开发者工作坊