Kurator v0.5.0: Crafting a Unified Multi-Cluster Backup and Storage Experience

Kurator is an open-source distributed cloud-native suite, designed to provide a one-stop solution for users looking to build their distributed cloud-native platform. In the latest version v0.5.0, Kurator has enhanced its capabilities in application backup and recovery, as well as storage management in multi-cluster environments, to meet the needs of complex deployments. This update mainly introduces two new features:

  • Unified Cluster Backup, Recovery, and Migration: Kurator now supports customized one-click backup and recovery of applications and resources across multiple clusters, with real-time monitoring of progress through a unified view. It also offers a one-click migration feature for cross-cluster resources.

  • Unified Distributed Storage: Kurator implements a consistent distributed storage solution, enabling users to easily implement block, file, and object storage in a multi-cluster environment with one-stop deployment.

Unified Cluster Backup, Recovery, and Migration

In the evolving landscape of multi-cloud and distributed environments, data security and recoverability have become paramount concerns for users. For enterprises, data loss can be a devastating blow, potentially leading to severe business interruptions and reputational damage. With Kubernetes becoming the industry standard, the complexity of data management increases alongside the growth in service numbers and cluster size, making efficient and flexible backup strategies crucial.

Facing the expanding demands and challenges, traditional backup tools often show limitations in multi-environment setups, failing to provide a seamless unified solution. Hence, Kurator’s unified backup solution was developed to offer a backup solution in this field. Based on Velero, Kurator provides a one-click operation experience, allowing users to customize backups and recover applications and resources across multiple clusters. With Kurator’s unified view feature, users can monitor the status and progress of backups in each cluster in real-time. Its coverage includes Kubernetes native resources like Pods, Deployments, Services, and PersistentVolumes, meeting the diverse data protection needs of modern enterprises.

Unified Backup

Kurator’s backup solution offers a variety of choices to suit different data protection needs in various scenarios. Its flexibility ensures that suitable backup strategies can be found for different business scenarios.

  • Instant Backup: In scenarios with frequent data changes, “instant backup” can quickly provide protection, ensuring the integrity of critical data at key moments.

  • Scheduled Backup: For data that changes less frequently but still requires persistent protection, “scheduled backup” can automatically perform backups at preset intervals, meeting compliance requirements and ensuring data security.

Additionally, Kurator provides a range of highly customizable backup options. For example, “specific cluster backup” allows operational teams to selectively back up specific clusters based on policies or specific needs. The “resource filtering” feature provides fine-grained control, enabling administrators to precisely define the scope of backups based on attributes such as resource names, namespaces, or labels.

These backup strategies’ diversity and automation capabilities provide stable and reliable data protection in the ever-changing business landscape.

Here is a practical example of unified backup:

apiVersion: backup.kurator.dev/v1alpha1
kind: Backup
metadata:
  ...
  name: select-labels
  namespace: default
spec:
  destination:
    fleet: quickstart
  policy:
    resourceFilter:
      labelSelector:
        matchLabels:
          app: busybox
    ttl: 720h
status:
  backupDetails:
  - backupNameInCluster: kurator-member1-backup-default-select-labels
    backupStatusInCluster:
      completionTimestamp: "2023-10-28T03:37:13Z"
      expiration: "2023-11-27T03:37:07Z"
      formatVersion: 1.1.0
      phase: Completed
      progress:
        itemsBackedUp: 1
        totalItems: 1
      startTimestamp: "2023-10-28T03:37:07Z"
      version: 1
    clusterKind: AttachedCluster
    clusterName: kurator-member1
  - backupNameInCluster: kurator-member2-backup-default-select-labels
    backupStatusInCluster:
      completionTimestamp: "2023-10-28T03:37:13Z"
      expiration: "2023-11-27T03:37:07Z"
      formatVersion: 1.1.0
      phase: Completed
      progress: {}
      startTimestamp: "2023-10-28T03:37:07Z"
      version: 1
    clusterKind: AttachedCluster
    clusterName: kurator-member2
  ...

Observing the spec configuration, the backup target is resources labeled with app:busybox across all clusters in the Fleet. By configuring policies in the spec, relevant resources are ensured to be backed up. The status allows real-time tracking of the backup task’s execution in each cluster, such as kurator-member1 and kurator-member2, maintaining transparency in operations.

🔗 For more examples and details, please refer to: Kurator Backup Documentation

Unified Recovery

Based on the backup data generated from unified backups, Kurator supports cross-cluster application and resource recovery through its unified recovery feature.

  • Instant Backup Recovery: Based on the backup data created from “instant backup,” you can quickly restore to a specified critical moment’s state.

  • Scheduled Backup Recovery: For “scheduled backup,” Kurator supports restoring backup data to the point of the last successful backup execution.

Similar to the backup function, Kurator also provides diverse and customizable options in recovery. For example, “specific cluster recovery” allows users to restore data to designated clusters without covering all clusters included in the backup. The “resource filtering” feature enables further selection of backup data, allowing selective restoration of needed data items. Users can define the scope of recovery based on backup names, namespaces, or labels, enhancing the recovery process’s flexibility and ensuring high precision.

Refer to the following operational example to understand how to use Kurator for unified recovery:

apiVersion: backup.kurator.dev/v1alpha1
kind: Restore
metadata:
  ...
  name: minimal
  namespace: default
spec:
  backupName: select-labels
status:
  restoreDetails:
  - clusterKind: AttachedCluster
    clusterName: kurator-member1
    restoreNameInCluster: kurator-member1-restore-default-minimal
    restoreStatusInCluster:
      completionTimestamp: "2023-10-28T09:24:07Z"
      phase: Completed
      progress:
        itemsRestored: 2
        totalItems: 2
      startTimestamp: "2023-10-28T09:24:05Z"
  - clusterKind: AttachedCluster
    clusterName: kurator-member2
    restoreNameInCluster: kurator-member2-restore-default-minimal
    restoreStatusInCluster:
      completionTimestamp: "2023-10-28T09:24:07Z"
      phase: Completed
      progress:
        itemsRestored: 2
        totalItems: 2
      startTimestamp: "2023-10-28T09:24:05Z"
  ...

By examining the recovery task’s spec configuration, we can confirm that this recovery operation targets the previously mentioned backup data labeled as select-labels. Here, the minimum configuration is used, not filtering during recovery but directly restoring according to the backup’s settings. In the status, the execution condition of the recovery task in each cluster can be tracked in real-time.

🔗 For more examples and details, please refer to: Kurator Recovery Documentation

Unified Migration

Unified migration aims to simplify the process of migrating applications and their resources from one cluster to several others. Users need to define a migrate-type resource configuration, specifying source clusters, target clusters, and related policies. Similar to Kurator’s unified backup and recovery functions, users can also perform a wealth of customized configurations.

Once the configuration is complete, Kurator’s corresponding controllers will automatically start the migration tasks. This series of tasks includes uploading resources from the source cluster to object storage and eventually migrating to the designated target clusters. The specific migration process can be referred to in the following diagram:

Compared to using Velero, Kurator provides a more integrated and clear migration process description. All necessary configuration details are centralized in a single migrate object, thus reducing the configuration burden as the number of target clusters increases. Additionally, Kurator automatically manages the entire process from creating backups to completing migration, simplifying operational procedures and reducing the risk of manual errors. Moreover, users can also monitor the migration progress in multiple clusters in real-time through this single object, staying informed of the latest migration status and ensuring the entire process runs as expected.

Here is a practical example of unified migration:

apiVersion: backup.kurator.dev/v1alpha1
kind: Migrate
metadata:
  ...
  name: select-labels
  namespace: default
spec:
  policy:
    resourceFilter:
      labelSelector:
        matchLabels:
          app: busybox
  sourceCluster:
    clusters:
    - kind: AttachedCluster
      name: kurator-member1
    fleet: quickstart
  targetCluster:
    clusters:
    - kind: AttachedCluster
      name: kurator-member2
    fleet: quickstart
status:
  conditions:
  - lastTransitionTime: "2023-10-28T15:55:23Z"
    status: "True"
    type: sourceReady
  phase: Completed
  sourceClusterStatus:
    backupNameInCluster: kurator-member1-migrate-default-select-labels
    backupStatusInCluster:
      completionTimestamp: "2023-10-28T15:55:18Z"
      expiration: "2023-11-27T15:55:13Z"
      formatVersion: 1.1.0
      phase: Completed
      progress: {}
      startTimestamp: "2023-10-28T15:55:13Z"
      version: 1
    clusterKind: AttachedCluster
    clusterName: kurator-member1
  targetClusterStatus:
  - clusterKind: AttachedCluster
    clusterName: kurator-member2
    restoreNameInCluster: kurator-member2-migrate-default-select-labels
    restoreStatusInCluster:
      completionTimestamp: "2023-10-28T15:56:00Z"
      phase: Completed
      startTimestamp: "2023-10-28T15:55:58Z"
  ...

In the spec configuration, the source cluster is set as kurator-member1, with the target cluster being kurator-member2, focusing the migration process only on resources containing the label app:busybox. In the status, the migration phase displays as Completed, indicating that the migration operation has been accomplished. sourceClusterStatus and targetClusterStatus provide details of the backup in the source cluster and the recovery in the target cluster, respectively.

🔗 For more details, please refer to: Kurator Migration Documentation

Unified Distributed Storage

Distributed storage, an indispensable part of modern cloud-native architectures, provides scalability and reliability of data. However, implementing a consistent distributed storage solution across different clusters often involves complex configurations and management tasks.

Kurator is committed to simplifying the deployment and management of distributed storage. Based on the leading open-source project Rook, Kurator supports easy and automated management of distributed storage across multiple clusters. This includes various storage types such as block, filesystem, and object storage, to meet the demands of various application scenarios.

Utilizing the Fleet plugin, Kurator offers a one-click cross-cluster deployment solution for distributed storage, simplifying configuration steps and significantly reducing the likelihood of configuration errors. The architecture is illustrated in the following diagram:

Here is an example of deploying multi-cluster distributed storage through the Fleet plugin:

apiVersion: fleet.kurator.dev/v1alpha1
kind: Fleet
metadata:
  name: quickstart
  namespace: default
spec:
  clusters:
    - name: kurator-member1
      kind: AttachedCluster
    - name: kurator-member2
      kind: AttachedCluster
  plugin:
    distributedStorage:
      storage:
        dataDirHostPath: /var/lib/rook
        monitor:
          count: 3
          labels:
            role: MonitorNodeLabel
        manager:
          count: 2
          labels:
            role: ManagerNodeLabel

In the spec, clusters indicate where the storage will be deployed. In the status, under the plugin configuration, distributedStorage signifies the installation of a distributed storage plugin. Additionally, dataDirHostPath defines the storage path, while monitor and manager configurations specify parameters for the Ceph components.

🔗 For more examples and details, please refer to: Kurator Distributed Storage Documentation

Community Contact Information