トップ 差分 一覧 ソース 検索 ヘルプ PDF RSS ログイン

FreeBSD gmirror 障害復旧手順例 201905

  • FreeBSD-10 でディスク障害が発生した。
  • FreeBSD gmirror GPT 3T HDD トリプルミラーでの障害
  • 障害復旧手順の記録
  • zfs ではない
  • GPT のバックアップ領域と gmirror の管理領域が重複するため、GPTではディスク丸ごと gmirror できない。
  • FreeBSD GPT gmirror 失敗
  • MBR だったらディスク丸ごと gmirror ができる。
  • 回避のため、GPTで複数のパーティションを作成し、それぞれのパーティションで gmirror を動作している。
  • 複数の gmirror が動作している。
  • ここでは、bootcode + freebsd-swap + freebsd-ufs

発生状況

4/4

SMART error (FailedOpenDevice) detected on host: ns4

This message was generated by the smartd daemon running on:
   host name:  ns4
   DNS domain: nona.to

The following warning/error was logged by the smartd daemon:
Device: /dev/ada0, unable to open ATA device

Device info:
TOSHIBA DT01ACA300, S/N:144K58KGS, WWN:5-000039-ff4d5e59e, FW:MX6OABB0, 3.00 TB

5/4

SMART error (FailedOpenDevice) detected on host: ns4

This message was generated by the smartd daemon running on:
   host name:  ns4
   DNS domain: nona.to

The following warning/error was logged by the smartd daemon:
Device: /dev/ada1, unable to open ATA device

Device info:
TOSHIBA DT01ACA300, S/N:Z7647YTAS, WWN:5-000039-fe6de1e60, FW:MX6OABB0, 3.00 TB
  • トリプルミラーの2台で発生
  • 発生状況を監視できていない。

過去の対応

作業概要

  • gmirror ada0 ada1 ada2 のトリプルミラー
  • ada0 ada1 が故障
  • ada0 を入れ替え
  • ada0 のパーティション設定
  • ada0 の起動コード書き込み
  • ミラー再開
  • ada1 を入れ替え
  • ada1 のパーティション設定
  • ada1 の起動コード書き込み
  • トリプルミラー再開

FreeBSD 正常時状況

  • /boot/loader.conf
geom_mirror_load="YES"
  • /etc/fstab
# Device        Mountpoint      FStype  Options Dump    Pass#
/dev/mirror/p2  /               ufs     rw      1       1
/dev/mirror/p3  none            swap    sw      0       0
  • gmirror 正常時
root@ns4:/home/hoge # gmirror status -s
mirror/p1  COMPLETE  ada0p1 (ACTIVE)
mirror/p1  COMPLETE  ada2p1 (ACTIVE)
mirror/p1  COMPLETE  ada1p1 (ACTIVE)
mirror/p2  COMPLETE  ada0p2 (ACTIVE)
mirror/p2  COMPLETE  ada2p2 (ACTIVE)
mirror/p2  COMPLETE  ada1p2 (ACTIVE)
mirror/p3  COMPLETE  ada0p3 (ACTIVE)
mirror/p3  COMPLETE  ada2p3 (ACTIVE)
mirror/p3  COMPLETE  ada1p3 (ACTIVE)

作業記録

状況確認

  • 過去のパーティション情報
root@ns4:/home/hoge # gpart list
Geom name: ada0
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 5860533134
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: ada0p1
  Mediasize: 65536 (64K)
  Sectorsize: 512
  Stripesize: 4096
  Stripeoffset: 0
  Mode: r1w1e1
  rawuuid: fd22e7d8-f409-11e3-9b6a-0022cfe54d22
  rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f
  label: (null)
  length: 65536
  offset: 20480
  type: freebsd-boot
  index: 1
  end: 167
  start: 40
2. Name: ada0p2
  Mediasize: 2996813361152 (2.7T)
  Sectorsize: 512
  Stripesize: 4096
  Stripeoffset: 0
  Mode: r1w1e1
  rawuuid: fd2317ba-f409-11e3-9b6a-0022cfe54d22
  rawtype: 516e7cb6-6ecf-11d6-8ff8-00022d09712b
  label: (null)
  length: 2996813361152
  offset: 86016
  type: freebsd-ufs
  index: 2
  end: 5853151263
  start: 168
3. Name: ada0p3
  Mediasize: 3779517440 (3.5G)
  Sectorsize: 512
  Stripesize: 4096
  Stripeoffset: 0
  Mode: r1w1e1
  rawuuid: fd25cafb-f409-11e3-9b6a-0022cfe54d22
  rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
  label: (null)
  length: 3779517440
  offset: 2996813447168
  type: freebsd-swap
  index: 3
  end: 5860533133
  start: 5853151264
Consumers:
1. Name: ada0
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r3w3e6

Geom name: ada2
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 5860533134
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: ada2p1
  Mediasize: 65536 (64K)
  Sectorsize: 512
  Stripesize: 4096
  Stripeoffset: 0
  Mode: r1w1e1
  rawuuid: 461f6d0d-f3c7-11e3-a841-0022cfe54d22
  rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f
  label: (null)
  length: 65536
  offset: 20480
  type: freebsd-boot
  index: 1
  end: 167
  start: 40
2. Name: ada2p2
  Mediasize: 2996813361152 (2.7T)
  Sectorsize: 512
  Stripesize: 4096
  Stripeoffset: 0
  Mode: r1w1e1
  rawuuid: 461fe080-f3c7-11e3-a841-0022cfe54d22
  rawtype: 516e7cb6-6ecf-11d6-8ff8-00022d09712b
  label: (null)
  length: 2996813361152
  offset: 86016
  type: freebsd-ufs
  index: 2
  end: 5853151263
  start: 168
3. Name: ada2p3
  Mediasize: 3779517440 (3.5G)
  Sectorsize: 512
  Stripesize: 4096
  Stripeoffset: 0
  Mode: r1w1e1
  rawuuid: 468c041d-f3c7-11e3-a841-0022cfe54d22
  rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
  label: (null)
  length: 3779517440
  offset: 2996813447168
  type: freebsd-swap
  index: 3
  end: 5860533133
  start: 5853151264
Consumers:
1. Name: ada2
  Mediasize: 3000592982016 (2.7T)
  Sectorsize: 512
  Stripesize: 4096
  Stripeoffset: 0
  Mode: r3w3e6
  • gmirror status 確認
root@ns4:/home/nonaka # gmirror status
     Name    Status  Components
mirror/p1  DEGRADED  ada2p1 (ACTIVE)
mirror/p2  DEGRADED  ada2p2 (ACTIVE)
mirror/p3  DEGRADED  ada2p3 (ACTIVE)
  • 超危険! 残り1台

GPT パーティション情報の確認

  • 生存 ada2 確認
gpart show ada2
root@ns4:/home/nonaka # gpart show ada2
=>        34  5860533101  ada2  GPT  (2.7T)
          34           6        - free -  (3.0K)
          40         128     1  freebsd-boot  (64K)
         168  5853151096     2  freebsd-ufs  (2.7T)
  5853151264     7381870     3  freebsd-swap  (3.5G)
  5860533134           1        - free -  (512B)

ada0 追加

物理ディスク ada0 接続

  • /var/log/messages
May 23 01:15:29 ns4 kernel: ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
May 23 01:15:29 ns4 kernel: ada0: <TOSHIBA DT01ACA300 MX6OABB0> ATA8-ACS SATA 3.x device
May 23 01:15:29 ns4 kernel: ada0: Serial Number Z4T4JT9GS
May 23 01:15:29 ns4 kernel: ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
May 23 01:15:29 ns4 kernel: ada0: Command Queueing enabled
May 23 01:15:29 ns4 kernel: ada0: 2861588MB (5860533168 512 byte sectors)
May 23 01:15:29 ns4 kernel: ada0: Previously was known as ad4

ada0 GPT パーティション作成

  • ada0 GPT パーティションを確認
root@ns4:/home/nonaka # gpart show ada0
=>        34  5860533101  ada0  GPT  (2.7T)
          34        2014        - free -  (1.0M)
        2048  5860529073     1  linux-lvm  (2.7T)
  5860531121        2014        - free -  (1.0M)
gpart delete -i 1 ada0
root@ns4:/home/nonaka # gpart delete -i 1 ada0
ada0p1 deleted
root@ns4:/home/nonaka # gpart show ada0
=>        34  5860533101  ada0  GPT  (2.7T)
          34  5860533101        - free -  (2.7T)
  • ada0 GPT 管理情報作成
gpart create -s gpt /dev/ada0
root@ns4:/home/nonaka # gpart create -s gpt /dev/ada0
gpart: geom 'ada0': File exists
  • GPT 管理情報は既に存在している。
  • ada0 GPT 初期状況確認
root@ns4:/home/nonaka # gpart show ada0
=>        34  5860533101  ada0  GPT  (2.7T)
          34  5860533101        - free -  (2.7T)

ada0 FreeBSD pmbr インストール

pmbr
Proactive Master Boot Record
root@ns4:/home/hoge # ls -l /boot/pmbr
-r--r--r--  1 root  wheel  512 Jan 17  2014 /boot/pmbr
root@ns4:/home/hoge # gpart bootcode -b /boot/pmbr ada0
bootcode written to ada0
  • MBR ツールによる破壊を防止するための疑似MBR情報+GPT起動コード

ada0 FreeBSD bootcode インストール

  • コマンド
ls -l /boot/gptboot
gpart add -b 40 -s 128 -t freebsd-boot ada0
gpart bootcode -p /boot/gptboot -i 1 ada0
  • ada2 GPT 情報より freebsd-boot 領域のサイズは 128
  • ログ
root@ns4:/home/nonaka # ls -l /boot/gptboot
-r--r--r--  1 root  wheel  16231 Feb 21  2018 /boot/gptboot
root@ns4:/home/nonaka # gpart show ada0
=>        34  5860533101  ada0  GPT  (2.7T)
          34  5860533101        - free -  (2.7T)
root@ns4:/home/nonaka # gpart add -b 40 -s 128 -t freebsd-boot ada0

ada0p1 added

root@ns4:/home/nonaka # gpart show ada0
=>        34  5860533101  ada0  GPT  (2.7T)
          34           6        - free -  (3.0K)
          40         128     1  freebsd-boot  (64K)
         168  5860532967        - free -  (2.7T)
  • (参考) GPT パーティションの除去
  • コマンド
gpart delete -i 1 ada1
  • ログ
root@ns4:/home/hoge # gpart show ada1
=>        34  5860533101  ada1  GPT  (2.7T)
          34           6        - free -  (3.0K)
          40  5860533088     1  freebsd-boot  (2.7T)
  5860533128           7        - free -  (3.5K)
root@ns4:/home/hoge # gpart delete -i 1 ada1
ada1p1 deleted
root@ns4:/home/hoge # gpart show ada1
=>        34  5860533101  ada1  GPT  (2.7T)
          34  5860533101        - free -  (2.7T)

ada0 GPT パーティション作成

  • コマンド
gpart add -t freebsd-ufs -s 5853151096 ada0
gpart add -t freebsd-swap -s 7381870 -a 2 ada0
  • ログ
root@ns4:/home/nonaka # gpart add -t freebsd-ufs -s 5853151096 ada0
ada0p2 added
root@ns4:/home/nonaka # gpart add -t freebsd-swap -s 7381870 -a 2 ada0
ada0p3 added
root@ns4:/home/nonaka # gpart show ada0
=>        34  5860533101  ada0  GPT  (2.7T)
          34           6        - free -  (3.0K)
          40         128     1  freebsd-boot  (64K)
         168  5853151096     2  freebsd-ufs  (2.7T)
  5853151264     7381870     3  freebsd-swap  (3.5G)
  5860533134           1        - free -  (512B)
  • ada2 と同じパーティション情報を ada0 に作成した。
  • gpart backup と gpart restore でパーティション情報を復旧するべき(要確認)

GPT パーティション状況確認

gpart show ada0
gpart show ada2
  • 同じパーティション情報であることを確認した。

gmirror 再構築

  • 単純に insert はできない。
root@ns4:/home/hoge # gmirror insert p1 /dev/ada0p1
gmirror: Not all disks connected.

DEGRADED 確認

root@ns4:/home/nonaka # gmirror status
     Name    Status  Components
mirror/p1  DEGRADED  ada2p1 (ACTIVE)
mirror/p2  DEGRADED  ada2p2 (ACTIVE)
mirror/p3  DEGRADED  ada2p3 (ACTIVE)

p1 DEGRADED 状況を解消する

root@ns4:/home/nonaka # gmirror forget p1
root@ns4:/home/nonaka # gmirror status
     Name    Status  Components
mirror/p1  COMPLETE  ada2p1 (ACTIVE)
mirror/p2  DEGRADED  ada2p2 (ACTIVE)
mirror/p3  DEGRADED  ada2p3 (ACTIVE)

p1 に ada0p1 を追加

root@ns4:/home/nonaka # gmirror insert p1 /dev/ada0p1
root@ns4:/home/nonaka # gmirror status
     Name    Status  Components
mirror/p1  COMPLETE  ada2p1 (ACTIVE)
                     ada0p1 (ACTIVE)
mirror/p2  DEGRADED  ada2p2 (ACTIVE)
mirror/p3  DEGRADED  ada2p3 (ACTIVE)

p3 に ada0p3 を追加

root@ns4:/home/nonaka # gmirror forget p3
root@ns4:/home/nonaka # gmirror status
     Name    Status  Components
mirror/p1  COMPLETE  ada2p1 (ACTIVE)
                     ada0p1 (ACTIVE)
mirror/p2  DEGRADED  ada2p2 (ACTIVE)
mirror/p3  COMPLETE  ada2p3 (ACTIVE)
root@ns4:/home/nonaka # gmirror insert p3 /dev/ada0p3
root@ns4:/home/nonaka # gmirror status
     Name    Status  Components
mirror/p1  COMPLETE  ada2p1 (ACTIVE)
                     ada0p1 (ACTIVE)
mirror/p2  DEGRADED  ada2p2 (ACTIVE)
mirror/p3  DEGRADED  ada2p3 (ACTIVE)
                     ada0p3 (SYNCHRONIZING, 4%)
root@ns4:/home/nonaka # gmirror status
     Name    Status  Components
mirror/p1  COMPLETE  ada2p1 (ACTIVE)
                     ada0p1 (ACTIVE)
mirror/p2  DEGRADED  ada2p2 (ACTIVE)
mirror/p3  DEGRADED  ada2p3 (ACTIVE)
                     ada0p3 (SYNCHRONIZING, 47%)
root@ns4:/home/nonaka # gmirror status
     Name    Status  Components
mirror/p1  COMPLETE  ada2p1 (ACTIVE)
                     ada0p1 (ACTIVE)
mirror/p2  DEGRADED  ada2p2 (ACTIVE)
mirror/p3  COMPLETE  ada2p3 (ACTIVE)
                     ada0p3 (ACTIVE)
  • p3 が DEGRADED から COMPLETE に変わった。
  • メッセージ確認
> grep GEOM_MIRROR /var/log/messages
May 23 01:50:37 ns4 kernel: GEOM_MIRROR: Device p1: rebuilding provider ada0p1.
May 23 01:50:37 ns4 kernel: GEOM_MIRROR: Device p1: rebuilding provider ada0p1 finished.
May 23 01:51:48 ns4 kernel: GEOM_MIRROR: Device p3: rebuilding provider ada0p3.
May 23 01:52:43 ns4 kernel: GEOM_MIRROR: Device p3: rebuilding provider ada0p3 finished.

p2 に ada0p2 を追加

root@ns4:/home/nonaka # gmirror status p2
     Name    Status  Components
mirror/p2  DEGRADED  ada2p2 (ACTIVE)
root@ns4:/home/nonaka # gmirror forget p2
root@ns4:/home/nonaka # gmirror status p2
     Name    Status  Components
mirror/p2  COMPLETE  ada2p2 (ACTIVE)
root@ns4:/home/nonaka # gmirror insert p2 /dev/ada0p2
root@ns4:/home/nonaka # gmirror status p2
    Name    Status  Components
mirror/p2  DEGRADED  ada2p2 (ACTIVE)
                     ada0p2 (SYNCHRONIZING, 4%)
  • 途中経過
root@ns4:/home/nonaka # gmirror status
     Name    Status  Components
mirror/p1  COMPLETE  ada2p1 (ACTIVE)
                     ada0p1 (ACTIVE)
mirror/p2  DEGRADED  ada2p2 (ACTIVE)
                     ada0p2 (SYNCHRONIZING, 57%)
mirror/p3  COMPLETE  ada2p3 (ACTIVE)
                     ada0p3 (ACTIVE)
  • 復旧完了状況確認
gmirror status
root@ns4:/home/nonaka # gmirror status
     Name    Status  Components
mirror/p1  COMPLETE  ada2p1 (ACTIVE)
                     ada0p1 (ACTIVE)
mirror/p2  COMPLETE  ada2p2 (ACTIVE)
                     ada0p2 (ACTIVE)
mirror/p3  COMPLETE  ada2p3 (ACTIVE)
                     ada0p3 (ACTIVE)
  • p2 が DEGRADED から COMPLETE に変わった。
  • /var/log/messages
May 23 01:57:29 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada0p2.
May 23 11:47:46 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada0p2 finished.
  • 2019年5月は10時間で完了
  • 2018年5月は9時間で完了
May 28 18:32:14 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada1p2.
May 29 03:29:06 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada1p2 finished.

ada1 ミラー追加

物理ディスク ada1 接続

  • /var/log/messages
May 23 02:40:12 ns4 kernel: ada1 at ahcich1 bus 0 scbus1 target 0 lun 0
May 23 02:40:12 ns4 kernel: ada1: <TOSHIBA MD04ACA300 FP2A> ATA8-ACS SATA 3.x device
May 23 02:40:12 ns4 kernel: ada1: Serial Number 74S3K1H4FSDA
May 23 02:40:12 ns4 kernel: ada1: 600.000MB/s transfers (SATA 3.x, UDMA5, PIO 8192bytes)
May 23 02:40:12 ns4 kernel: ada1: Command Queueing enabled
May 23 02:40:12 ns4 kernel: ada1: 2861588MB (5860533168 512 byte sectors)
May 23 02:40:12 ns4 kernel: ada1: Previously was known as ad6

ada1 GPT パーティション作成

  • ada1 GPT パーティションを確認
gpart show ada1
root@ns4:/home/nonaka # gpart show ada1
=>        34  5860533101  ada1  GPT  (2.7T)
          34        2014        - free -  (1.0M)
        2048  5860529073     1  linux-lvm  (2.7T)
  5860531121        2014        - free -  (1.0M)
gpart delete -i 1 ada1
root@ns4:/home/nonaka # gpart delete -i 1 ada1
ada1p1 deleted
root@ns4:/home/nonaka # gpart show ada1
=>        34  5860533101  ada1  GPT  (2.7T)
          34  5860533101        - free -  (2.7T)
  • ada0 GPT 管理情報作成
gpart destroy /dev/ada1
gpart create -s gpt /dev/ada1
root@ns4:/home/nonaka # gpart destroy /dev/ada1
ada1 destroyed
root@ns4:/home/nonaka # gpart show ada1
gpart: No such geom: ada1.
root@ns4:/home/nonaka # gpart create -s gpt /dev/ada1
ada1 created
  • ada0 GPT 初期状況確認
root@ns4:/home/nonaka # gpart show ada1
=>        34  5860533101  ada1  GPT  (2.7T)
          34  5860533101        - free -  (2.7T)

ada0 FreeBSD pmbr インストール

pmbr
Proactive Master Boot Record
root@ns4:/home/nonaka # ls -l /boot/pmbr
-r--r--r--  1 root  wheel  512 Jan 17  2014 /boot/pmbr
root@ns4:/home/nonaka # gpart bootcode -b /boot/pmbr ada1
bootcode written to ada1
  • MBR ツールによる破壊を防止するための疑似MBR情報+GPT起動コード

ada0 FreeBSD bootcode インストール

  • コマンド
ls -l /boot/gptboot
gpart add -b 40 -s 128 -t freebsd-boot ada1
gpart bootcode -p /boot/gptboot -i 1 ada1
  • ada2 GPT 情報より freebsd-boot 領域のサイズは 128
  • ログ
root@ns4:/home/nonaka # ls -l /boot/gptboot
-r--r--r--  1 root  wheel  16231 Feb 21  2018 /boot/gptboot
root@ns4:/home/nonaka # gpart add -b 40 -s 128 -t freebsd-boot ada1
ada1p1 added
root@ns4:/home/nonaka # gpart show ada1
=>        34  5860533101  ada1  GPT  (2.7T)
          34           6        - free -  (3.0K)
          40         128     1  freebsd-boot  (64K)
         168  5860532967        - free -  (2.7T)
root@ns4:/home/nonaka # gpart bootcode -p /boot/gptboot -i 1 ada1
partcode written to ada1p1
  • (参考) GPT パーティションの除去
  • コマンド
gpart delete -i 1 ada1
  • ログ
root@ns4:/home/hoge # gpart show ada1
=>        34  5860533101  ada1  GPT  (2.7T)
          34           6        - free -  (3.0K)
          40  5860533088     1  freebsd-boot  (2.7T)
  5860533128           7        - free -  (3.5K)
root@ns4:/home/hoge # gpart delete -i 1 ada1
ada1p1 deleted
root@ns4:/home/hoge # gpart show ada1
=>        34  5860533101  ada1  GPT  (2.7T)
          34  5860533101        - free -  (2.7T)

ada1 GPT パーティション作成

  • コマンド
gpart add -t freebsd-ufs -s 5853151096 ada1
gpart add -t freebsd-swap -s 7381870 -a 2 ada1
  • ログ
root@ns4:/home/nonaka # gpart add -t freebsd-ufs -s 5853151096 ada1
ada1p2 added
root@ns4:/home/nonaka # gpart add -t freebsd-swap -s 7381870 -a 2 ada1
ada1p3 added
root@ns4:/home/nonaka # gpart show ada1
=>        34  5860533101  ada1  GPT  (2.7T)
          34           6        - free -  (3.0K)
          40         128     1  freebsd-boot  (64K)
         168  5853151096     2  freebsd-ufs  (2.7T)
  5853151264     7381870     3  freebsd-swap  (3.5G)
  5860533134           1        - free -  (512B)
  • ada2 と同じパーティション情報を ada1 に作成した。
  • gpart backup と gpart restore でパーティション情報を復旧するべき(要確認)

GPT パーティション状況確認

gpart show ada1
gpart show ada2
  • 同じパーティション情報であることを確認した。

gmirror 追加 hdd3台目

  • (参考) DEGRADED の場合は単純に insert はできない。
root@ns4:/home/hoge # gmirror insert p1 /dev/ada1p1
gmirror: Not all disks connected.

gmirror 状況確認

root@ns4:/home/nonaka # gmirror status
     Name    Status  Components
mirror/p1  COMPLETE  ada2p1 (ACTIVE)
                     ada0p1 (ACTIVE)
mirror/p2  COMPLETE  ada2p2 (ACTIVE)
                     ada0p2 (ACTIVE)
mirror/p3  COMPLETE  ada2p3 (ACTIVE)
                     ada0p3 (ACTIVE)
  • DEGRADED の gmirror はない

p1 に ada0p1 を追加

gmirror status p1
gmirror insert p1 /dev/ada1p1
gmirror status p1
root@ns4:/home/nonaka # gmirror status p1
     Name    Status  Components
mirror/p1  COMPLETE  ada2p1 (ACTIVE)
                     ada0p1 (ACTIVE)
root@ns4:/home/nonaka # gmirror insert p1 /dev/ada1p1
root@ns4:/home/nonaka # gmirror status p1
     Name    Status  Components
mirror/p1  COMPLETE  ada2p1 (ACTIVE)
                     ada0p1 (ACTIVE)
                     ada1p1 (ACTIVE)
  • p1 が2台のgmirror から3台の gmirror に変わった。
  • /var/log/messages
May 23 14:53:28 ns4 kernel: GEOM_MIRROR: Device p1: rebuilding provider ada1p1.
May 23 14:53:28 ns4 kernel: GEOM_MIRROR: Device p1: rebuilding provider ada1p1 finished.

p3 に ada1p3 を追加

gmirror status p3
gmirror insert p3 /dev/ada1p3
gmirror status p3
root@ns4:/home/nonaka # gmirror status p3
     Name    Status  Components
mirror/p3  COMPLETE  ada2p3 (ACTIVE)
                     ada0p3 (ACTIVE)
root@ns4:/home/nonaka # gmirror insert p3 /dev/ada1p3
root@ns4:/home/nonaka # gmirror status p3
     Name    Status  Components
mirror/p3  DEGRADED  ada2p3 (ACTIVE)
                     ada0p3 (ACTIVE)
                     ada1p3 (SYNCHRONIZING, 87%)
root@ns4:/home/nonaka # gmirror status p3
     Name    Status  Components
mirror/p3  COMPLETE  ada2p3 (ACTIVE)
                     ada0p3 (ACTIVE)
                     ada1p3 (ACTIVE)
  • /var/log/messages
May 23 14:56:35 ns4 kernel: GEOM_MIRROR: Device p3: rebuilding provider ada1p3.
May 23 14:57:29 ns4 kernel: GEOM_MIRROR: Device p3: rebuilding provider ada1p3 finished.
  • p3 が2台のgmirror から3台の gmirror に変わった。

p2 に ada1p2 を追加

gmirror status p2
gmirror insert p2 /dev/ada1p2
gmirror status p2
root@ns4:/home/nonaka # gmirror status p2
     Name    Status  Components
mirror/p2  COMPLETE  ada2p2 (ACTIVE)
                     ada0p2 (ACTIVE)
root@ns4:/home/nonaka # gmirror insert p2 /dev/ada1p2
root@ns4:/home/nonaka # gmirror status p2
     Name    Status  Components
mirror/p2  DEGRADED  ada2p2 (ACTIVE)
                     ada0p2 (ACTIVE)
                     ada1p2 (SYNCHRONIZING, 0%)
  • 途中経過
root@ns4:/home/nonaka # gmirror status
     Name    Status  Components
mirror/p1  COMPLETE  ada2p1 (ACTIVE)
                     ada0p1 (ACTIVE)
mirror/p2  DEGRADED  ada2p2 (ACTIVE)
                     ada0p2 (SYNCHRONIZING, 57%)
mirror/p3  COMPLETE  ada2p3 (ACTIVE)
                     ada0p3 (ACTIVE)
  • 復旧完了状況確認
gmirror status
root@ns4:/home/nonaka # gmirror status p2
     Name    Status  Components
mirror/p2  DEGRADED  ada2p2 (ACTIVE)
                     ada0p2 (ACTIVE)
                     ada1p2 (SYNCHRONIZING, 1%)
  • p2 が DEGRADED から COMPLETE に変わった。
  • /var/log/messages
  • 2019年5月は10時間で完了
  • 2018年5月は9時間で完了
May 28 18:32:14 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada1p2.
May 29 03:29:06 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada1p2 finished.

起動確認

  • 3台のHDDを入れ替えて3台とも起動できることを確認する。
  • pmbr は正常か? bootcode は正常か? 確認する。
  • shutdown
  • HDD1-->HDD2-->HDD3-->HDD1
  • 起動確認
  • shutdown
  • HDD1-->HDD2-->HDD3-->HDD1
  • 起動確認
  • shutdown
  • HDD1-->HDD2-->HDD3-->HDD1
  • 起動確認