OpenStack Ansible Managed Ceph

Deploy: OpenStack-Ansible
2026-04-17T07:12:20+00:00
Deploy and manage Ceph clusters with OpenStack Ansible using YAML configurations and dedicated playbooks.

OpenStack Ansible Managed Ceph

OpenStack-Ansible deploys Ceph by configuring host groups in openstack_user_config.yml, defining variables in user_variables_ceph.yml, and executing the ceph-install.yml playbook.

Prerequisites

  • OSA Version: 2024.2 (Dalmatian) or later
  • Ceph Nodes: Minimum 3 nodes (Monitors + OSDs)
  • OS: Ubuntu 22.04 LTS (Jammy)
  • Storage: 1+ unused block device per OSD node (e.g., /dev/sdb)
  • Network: Dedicated storage network recommended; public/cluster subnets defined

Step 1: Define Ceph Hosts

Configure host groups in /etc/openstack_deploy/openstack_user_config.yml.

ceph-mon_hosts:
  ceph01:
    ip: 172.29.236.21
  ceph02:
    ip: 172.29.236.22
  ceph03:
    ip: 172.29.236.23

ceph-osd_hosts:
  ceph01:
    ip: 172.29.236.21
    container_vars:
      devices:
        - /dev/sdb
        - /dev/sdc
  ceph02:
    ip: 172.29.236.22
    container_vars:
      devices:
        - /dev/sdb
        - /dev/sdc
  ceph03:
    ip: 172.29.236.23
    container_vars:
      devices:
        - /dev/sdb
        - /dev/sdc

Step 2: Configure Ceph Variables

Define deployment parameters in /etc/openstack_deploy/user_variables_ceph.yml.

ceph_pkg_source: distro
ceph_stable_release: reef
ceph_public_network: 172.29.236.0/22
ceph_cluster_network: 172.29.244.0/22
osd_scenario: lvm
osd_objectstore: bluestore
ceph_conf_overrides:
  global:
    osd_pool_default_size: 3
    osd_pool_default_min_size: 2
    osd_pool_default_pg_num: 64
    osd_pool_default_pgp_num: 64

Step 3: Enable Ceph for OpenStack Services

Configure OpenStack services to utilize Ceph pools in /etc/openstack_deploy/user_variables.yml.

glance_default_store: rbd
glance_rbd_store_pool: images
cinder_backends:
  rbd:
    volume_driver: cinder.volume.drivers.rbd.RBDDriver
    rbd_pool: volumes
    rbd_ceph_conf: /etc/ceph/ceph.conf
    rbd_user: cinder
    volume_backend_name: rbd
    report_discard_supported: true
    rbd_flatten_volume_from_snapshot: true
    rbd_max_clone_depth: 3
nova_libvirt_images_rbd_pool: vms
nova_libvirt_images_rbd_ceph_conf: /etc/ceph/ceph.conf
nova_libvirt_images_rbd_user: nova

Step 4: Deploy the Ceph Cluster

Execute the dedicated Ceph installation playbook from the OSA directory.

cd /opt/openstack-ansible
openstack-ansible playbooks/ceph-install.yml

Step 5: Deploy OpenStack Services

Run the standard OpenStack deployment playbooks to consume the distributed Ceph configuration.

openstack-ansible playbooks/setup-hosts.yml
openstack-ansible playbooks/setup-infrastructure.yml
openstack-ansible playbooks/setup-openstack.yml

Step 6: Verify the Deployment

Validate Ceph cluster health and OpenStack integration.

sudo ceph -s
sudo ceph osd tree
sudo ceph osd pool ls detail
openstack image create --disk-format raw --container-format bare --file cirros.img cirros-test
openstack volume create --size 10 test-vol
rbd -p images ls
rbd -p volumes ls

Scaling and Operations

Add a new OSD host: Add the host to ceph-osd_hosts in openstack_user_config.yml and run openstack-ansible playbooks/ceph-install.yml --limit new-ceph-host.

Add disks to an existing host: Update the devices list in openstack_user_config.yml and run openstack-ansible playbooks/ceph-install.yml --limit existing-host.

Replace failed OSD: Run ceph osd out <id>, replace the disk, and run openstack-ansible playbooks/ceph-install.yml --limit <host>.

Upgrade Ceph: Update ceph_stable_release in user_variables_ceph.yml and run openstack-ansible playbooks/ceph-install.yml.

Troubleshooting

  • OSD not created: Verify disk is unused (lsblk) and wipe partitions (wipefs -a /dev/sdX).
  • Monitor quorum failure: Verify NTP sync (chronyc sources) and network connectivity between Ceph nodes.
  • Playbook failure: Run with verbose output (openstack-ansible playbooks/ceph-install.yml -vvv).
  • Pool missing: Verify ceph_conf_overrides and pool definitions in user_variables_ceph.yml.