トップ差分一覧ソース検索ヘルプ PDF RSS ログイン

FreeBSD gmirror 障害復旧手順例 202008

FreeBSD-11 でディスク障害が発生した。
FreeBSD gmirror GPT 3T HDD トリプルミラーでの障害
セクタエラー
障害復旧手順の記録

zfs ではない
GPT のバックアップ領域と gmirror の管理領域が重複するため、GPTではディスク丸ごと gmirror できない。
FreeBSD GPT gmirror 失敗
MBR だったらディスク丸ごと gmirror ができる。
回避のため、GPTで複数のパーティションを作成し、それぞれのパーティションで gmirror を動作している。
複数の gmirror が動作している。
ここでは、bootcode + freebsd-swap + freebsd-ufs

発生状況

4/4

SMART error (FailedOpenDevice) detected on host: ns4

This message was generated by the smartd daemon running on:
   host name:  ns4
   DNS domain: nona.to

The following warning/error was logged by the smartd daemon:
Device: /dev/ada0, unable to open ATA device

Device info:
TOSHIBA DT01ACA300, S/N:144K58KGS, WWN:5-000039-ff4d5e59e, FW:MX6OABB0, 3.00 TB

5/4

TOP ↑ ↓

SMART error (FailedOpenDevice) detected on host: ns4

This message was generated by the smartd daemon running on:
   host name:  ns4
   DNS domain: nona.to

The following warning/error was logged by the smartd daemon:
Device: /dev/ada1, unable to open ATA device

Device info:
TOSHIBA DT01ACA300, S/N:Z7647YTAS, WWN:5-000039-fe6de1e60, FW:MX6OABB0, 3.00 TB

I/Oエラー

root@ns4:/home/hoge # time dd if=/dev/ada1 of=/dev/null bs=1048576 skip=1295380
dd: /dev/ada1: Input/output error
5+0 records in
5+0 records out
5242880 bytes transferred in 14.679095 secs (357166 bytes/sec)
0.000u 0.001s 0:14.68 0.0%      0+0k 43+0io 0pf+0w

1.29GB 付近でIOエラー

過去の対応

TOP ↑ ↓

作業概要

TOP ↑ ↓

gmirror ada0 ada1 ada2 のトリプルミラー
ada1 が故障

ada1 を入れ替え
ada1 のパーティション設定
ada1 の起動コード書き込み
トリプルミラー再開

FreeBSD 正常時状況

TOP ↑ ↓

/boot/loader.conf

geom_mirror_load="YES"

/etc/fstab

# Device        Mountpoint      FStype  Options Dump    Pass#
/dev/mirror/p2  /               ufs     rw      1       1
/dev/mirror/p3  none            swap    sw      0       0

gmirror 正常時

root@ns4:/home/hoge # gmirror status -s
mirror/p1  COMPLETE  ada0p1 (ACTIVE)
mirror/p1  COMPLETE  ada2p1 (ACTIVE)
mirror/p1  COMPLETE  ada1p1 (ACTIVE)
mirror/p2  COMPLETE  ada0p2 (ACTIVE)
mirror/p2  COMPLETE  ada2p2 (ACTIVE)
mirror/p2  COMPLETE  ada1p2 (ACTIVE)
mirror/p3  COMPLETE  ada0p3 (ACTIVE)
mirror/p3  COMPLETE  ada2p3 (ACTIVE)
mirror/p3  COMPLETE  ada1p3 (ACTIVE)

作業記録

TOP ↑ ↓

状況確認

TOP ↑ ↓

パーティション情報

root@ns4:/home/hoge # gpart list
Geom name: ada0
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 5860533134
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: ada0p1
   Mediasize: 65536 (64K)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
   efimedia: HD(1,GPT,eaa06e43-7cb0-11e9-8d65-6805ca0d3e8b,0x28,0x80)
   rawuuid: eaa06e43-7cb0-11e9-8d65-6805ca0d3e8b
   rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f
   label: (null)
   length: 65536
   offset: 20480
   type: freebsd-boot
   index: 1
   end: 167
   start: 40
2. Name: ada0p2
   Mediasize: 2996813361152 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
   efimedia: HD(2,GPT,35c63824-7cb1-11e9-8d65-6805ca0d3e8b,0xa8,0x15cdfff78)
   rawuuid: 35c63824-7cb1-11e9-8d65-6805ca0d3e8b
   rawtype: 516e7cb6-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 2996813361152
   offset: 86016
   type: freebsd-ufs
   index: 2
   end: 5853151263
   start: 168
3. Name: ada0p3
   Mediasize: 3779517440 (3.5G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
   efimedia: HD(3,GPT,46597fa0-7cb1-11e9-8d65-6805ca0d3e8b,0x15ce00020,0x70a36e)
   rawuuid: 46597fa0-7cb1-11e9-8d65-6805ca0d3e8b
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 3779517440
   offset: 2996813447168
   type: freebsd-swap
   index: 3
   end: 5860533133
   start: 5853151264
Consumers:
1. Name: ada0
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r3w3e6

Geom name: ada1
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 5860533134
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: ada1p1
   Mediasize: 65536 (64K)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
   efimedia: HD(1,GPT,afd1aa88-7d1d-11e9-8d65-6805ca0d3e8b,0x28,0x80)
   rawuuid: afd1aa88-7d1d-11e9-8d65-6805ca0d3e8b
   rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f
   label: (null)
   length: 65536
   offset: 20480
   type: freebsd-boot
   index: 1
   end: 167
   start: 40
2. Name: ada1p2
   Mediasize: 2996813361152 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
   efimedia: HD(2,GPT,55594ddf-7d1e-11e9-8d65-6805ca0d3e8b,0xa8,0x15cdfff78)
   rawuuid: 55594ddf-7d1e-11e9-8d65-6805ca0d3e8b
   rawtype: 516e7cb6-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 2996813361152
   offset: 86016
   type: freebsd-ufs
   index: 2
   end: 5853151263
   start: 168
3. Name: ada1p3
   Mediasize: 3779517440 (3.5G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
   efimedia: HD(3,GPT,59e22286-7d1e-11e9-8d65-6805ca0d3e8b,0x15ce00020,0x70a36e)
   rawuuid: 59e22286-7d1e-11e9-8d65-6805ca0d3e8b
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 3779517440
   offset: 2996813447168
   type: freebsd-swap
   index: 3
   end: 5860533133
   start: 5853151264
Consumers:
1. Name: ada1
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r3w3e6

Geom name: ada2
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 5860533134
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: ada2p1
   Mediasize: 65536 (64K)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
   efimedia: HD(1,GPT,461f6d0d-f3c7-11e3-a841-0022cfe54d22,0x28,0x80)
   rawuuid: 461f6d0d-f3c7-11e3-a841-0022cfe54d22
   rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f
   label: (null)
   length: 65536
   offset: 20480
   type: freebsd-boot
   index: 1
   end: 167
   start: 40
2. Name: ada2p2
   Mediasize: 2996813361152 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
   efimedia: HD(2,GPT,461fe080-f3c7-11e3-a841-0022cfe54d22,0xa8,0x15cdfff78)
   rawuuid: 461fe080-f3c7-11e3-a841-0022cfe54d22
   rawtype: 516e7cb6-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 2996813361152
   offset: 86016
   type: freebsd-ufs
   index: 2
   end: 5853151263
   start: 168
3. Name: ada2p3
   Mediasize: 3779517440 (3.5G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
   efimedia: HD(3,GPT,468c041d-f3c7-11e3-a841-0022cfe54d22,0x15ce00020,0x70a36e)
   rawuuid: 468c041d-f3c7-11e3-a841-0022cfe54d22
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 3779517440
   offset: 2996813447168
   type: freebsd-swap
   index: 3
   end: 5860533133
   start: 5853151264
Consumers:
1. Name: ada2
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r3w3e6

gmirror status 確認

root@ns4:/home/hoge # gmirror status
     Name    Status  Components
mirror/p1  COMPLETE  ada0p1 (ACTIVE)
                     ada1p1 (ACTIVE)
                     ada2p1 (ACTIVE)
mirror/p2  COMPLETE  ada0p2 (ACTIVE)
                     ada1p2 (ACTIVE)
                     ada2p2 (ACTIVE)
mirror/p3  COMPLETE  ada0p3 (ACTIVE)
                     ada1p3 (ACTIVE)
                     ada2p3 (ACTIVE)

正常に見えるが、ada1 でIOエラー

root@ns4:/home/hoge # time dd if=/dev/ada1 of=/dev/null bs=1048576 skip=1295380
dd: /dev/ada1: Input/output error
5+0 records in
5+0 records out
5242880 bytes transferred in 14.679095 secs (357166 bytes/sec)
0.000u 0.001s 0:14.68 0.0%      0+0k 43+0io 0pf+0w

ada1 1.29GB 付近でIOエラー

GPT パーティション情報の確認

TOP ↑ ↓

ada0 ada2 確認

gpart show ada0
gpart show ada1
gpart show ada2

root@ns4:/home/hoge # gpart show ada0
=>        34  5860533101  ada0  GPT  (2.7T)
          34           6        - free -  (3.0K)
          40         128     1  freebsd-boot  (64K)
         168  5853151096     2  freebsd-ufs  (2.7T)
  5853151264     7381870     3  freebsd-swap  (3.5G)
  5860533134           1        - free -  (512B)

root@ns4:/home/hoge # gpart show ada1
=>        34  5860533101  ada1  GPT  (2.7T)
          34           6        - free -  (3.0K)
          40         128     1  freebsd-boot  (64K)
         168  5853151096     2  freebsd-ufs  (2.7T)
  5853151264     7381870     3  freebsd-swap  (3.5G)
  5860533134           1        - free -  (512B)

root@ns4:/home/hoge # gpart show ada2
=>        34  5860533101  ada2  GPT  (2.7T)
          34           6        - free -  (3.0K)
          40         128     1  freebsd-boot  (64K)
         168  5853151096     2  freebsd-ufs  (2.7T)
  5853151264     7381870     3  freebsd-swap  (3.5G)
  5860533134           1        - free -  (512B)

ada1 交換

TOP ↑ ↓

物理ディスク ada1 除去

TOP ↑ ↓

/var/log/messages

Aug 23 00:36:45 ns4 kernel: ada1 at ahcich1 bus 0 scbus1 target 0 lun 0
Aug 23 00:36:45 ns4 kernel: ada1: <TOSHIBA MD04ACA300 FP2A> s/n 74S3K1H4FSDA detached
Aug 23 00:36:45 ns4 kernel: GEOM_MIRROR: Device p3: provider ada1p3 disconnected.GEOM_MIRROR
Aug 23 00:36:45 ns4 kernel: : Device p1: provider ada1p1 disconnected.
Aug 23 00:36:45 ns4 kernel: GEOM_MIRROR: Device p2: provider ada1p2 disconnected.
Aug 23 00:36:45 ns4 kernel: (ada1:ahcich1:0:0:0): Periph destroyed

root@ns4:/home/hoge # gmirror status
     Name    Status  Components
mirror/p1  DEGRADED  ada0p1 (ACTIVE)
                     ada2p1 (ACTIVE)
mirror/p2  DEGRADED  ada0p2 (ACTIVE)
                     ada2p2 (ACTIVE)
mirror/p3  DEGRADED  ada0p3 (ACTIVE)
                     ada2p3 (ACTIVE)

物理ディスク ada1 接続

TOP ↑ ↓

/var/log/messages

Aug 23 00:40:24 ns4 kernel: ada1 at ahcich1 bus 0 scbus1 target 0 lun 0
Aug 23 00:40:24 ns4 kernel: ada1: <TOSHIBA DT01ACA300 MX6OABB0> ATA8-ACS SATA 3.x device
Aug 23 00:40:24 ns4 kernel: ada1: Serial Number 29O6V7GAS
Aug 23 00:40:24 ns4 kernel: ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
Aug 23 00:40:24 ns4 kernel: ada1: Command Queueing enabled
Aug 23 00:40:24 ns4 kernel: ada1: 2861588MB (5860533168 512 byte sectors)
Aug 23 00:40:24 ns4 kernel: GEOM: ada1: the secondary GPT header is not in the last LBA.
Aug 23 00:40:24 ns4 kernel: GEOM: diskid/DISK-29O6V7GAS: the secondary GPT header is not in the last LBA.

ada1 GPT パーティション作成

TOP ↑ ↓

ada1 GPT パーティションを確認

root@ns4:/home/hoge # gpart show ada1
=>        34  5860533100  ada1  GPT  (2.7T) [CORRUPT]
          34       65501        - free -  (32M)
       65535  5860467375     1  linux-data  (2.7T)
  5860532910         224        - free -  (112K)

Linux のパーティションが存在する。

gpart delete -i 1 ada1

root@ns4:/home/hoge # gpart delete -i 1 ada1
gpart: table 'ada1' is corrupt: Operation not permitted

root@ns4:/home/hoge # gpart recover /dev/ada1
ada1 recovered

root@ns4:/home/hoge # gpart show ada1
=>        40  5860533088  ada1  GPT  (2.7T)
          40       65495        - free -  (32M)
       65535  5860467375     1  linux-data  (2.7T)
  5860532910         218        - free -  (109K)

root@ns4:/home/hoge # gpart delete -i 1 ada1
ada1p1 deleted

root@ns4:/home/hoge # gpart show ada1
=>        40  5860533088  ada1  GPT  (2.7T)
          40  5860533088        - free -  (2.7T)

ada0 GPT 管理情報作成

gpart create -s gpt /dev/ada1

root@ns4:/home/hoge # gpart create -s gpt /dev/ada1
gpart: geom 'ada1': File exists

GPT 管理情報は既に存在している。

ada1 GPT 初期状況確認

root@ns4:/home/hoge # gpart show ada1
=>        40  5860533088  ada1  GPT  (2.7T)
          40  5860533088        - free -  (2.7T)

ada0 FreeBSD pmbr インストール

TOP ↑ ↓

pmbr: Proactive Master Boot Record

root@ns4:/home/hoge # ls -l /boot/pmbr
root@ns4:/home/hoge # gpart bootcode -b /boot/pmbr ada1
bootcode written to ada1

MBR ツールによる破壊を防止するための疑似MBR情報+GPT起動コード

ada1 FreeBSD bootcode インストール

TOP ↑ ↓

コマンド

ls -l /boot/gptboot
gpart add -b 40 -s 128 -t freebsd-boot ada1
gpart bootcode -p /boot/gptboot -i 1 ada1

ada0 GPT 情報より freebsd-boot 領域のサイズは 128
ログ

root@ns4:/home/hoge # ls -l /boot/gptboot
-r--r--r--  1 root  wheel  59074 Aug 10  2019 /boot/gptboot
root@ns4:/home/hoge # gpart show ada1
=>        40  5860533088  ada1  GPT  (2.7T)
          40  5860533088        - free -  (2.7T)

root@ns4:/home/hoge # gpart add -b 40 -s 128 -t freebsd-boot ada1
ada1p1 added
root@ns4:/home/hoge # gpart show ada1
=>        40  5860533088  ada1  GPT  (2.7T)
          40         128     1  freebsd-boot  (64K)
         168  5860532960        - free -  (2.7T)

(参考) GPT パーティションの除去
コマンド

gpart delete -i 1 ada1

ログ

root@ns4:/home/hoge # gpart show ada1
=>        34  5860533101  ada1  GPT  (2.7T)
          34           6        - free -  (3.0K)
          40  5860533088     1  freebsd-boot  (2.7T)
  5860533128           7        - free -  (3.5K)
root@ns4:/home/hoge # gpart delete -i 1 ada1
ada1p1 deleted
root@ns4:/home/hoge # gpart show ada1
=>        34  5860533101  ada1  GPT  (2.7T)
          34  5860533101        - free -  (2.7T)

ada0 GPT パーティション作成

TOP ↑ ↓

コマンド

gpart add -t freebsd-ufs -s 5853151096 ada1
gpart add -t freebsd-swap -s   7381870 -a 2 ada1

ログ

root@ns4:/home/hoge # gpart add -t freebsd-ufs -s 5853151096 ada1
ada1p2 added

root@ns4:/home/hoge # gpart add -t freebsd-swap -s   7381870 -a 2 ada1
gpart: autofill: No space left on device
root@ns4:/home/hoge # gpart show ada1
=>        40  5860533088  ada1  GPT  (2.7T)
          40         128     1  freebsd-boot  (64K)
         168  5853151096     2  freebsd-ufs  (2.7T)
  5853151264     7381864        - free -  (3.5G)

root@ns4:/home/hoge # gpart add -t freebsd-swap -s 7381864 -a 2 ada1
ada1p3 added
root@ns4:/home/hoge # gpart show ada1
=>        40  5860533088  ada1  GPT  (2.7T)
          40         128     1  freebsd-boot  (64K)
         168  5853151096     2  freebsd-ufs  (2.7T)
  5853151264     7381864     3  freebsd-swap  (3.5G)

ada2 と*ほぼ*同じパーティション情報を ada1 に作成した。
freebsd-swap のサイズが小さいので復旧できていない。別途対応

gpart backup と gpart restore でパーティション情報を復旧するべき(要確認)

GPT パーティション状況確認

TOP ↑ ↓

gpart show ada0
gpart show ada2

同じパーティション情報であることを確認した。

gmirror 再構築

TOP ↑ ↓

単純に insert はできない。

root@ns4:/home/hoge # gmirror insert p1 /dev/ada1p1
gmirror: Not all disks connected.

DEGRADED　確認

TOP ↑ ↓

root@ns4:/home/hoge # gmirror status
     Name    Status  Components
mirror/p1  DEGRADED  ada0p1 (ACTIVE)
                     ada2p1 (ACTIVE)
mirror/p2  DEGRADED  ada0p2 (ACTIVE)
                     ada2p2 (ACTIVE)
mirror/p3  DEGRADED  ada0p3 (ACTIVE)
                     ada2p3 (ACTIVE)

p1 DEGRADED 状況を解消する

TOP ↑ ↓

root@ns4:/home/hoge # gmirror forget p1
root@ns4:/home/hoge # gmirror status
     Name    Status  Components
mirror/p1  COMPLETE  ada0p1 (ACTIVE)
                     ada2p1 (ACTIVE)
mirror/p2  DEGRADED  ada0p2 (ACTIVE)
                     ada2p2 (ACTIVE)
mirror/p3  DEGRADED  ada0p3 (ACTIVE)
                     ada2p3 (ACTIVE)

p1 に ada1p1 を追加

TOP ↑ ↓

root@ns4:/home/hoge # gmirror insert p1 /dev/ada1p1
root@ns4:/home/hoge # gmirror status
     Name    Status  Components
mirror/p1  COMPLETE  ada0p1 (ACTIVE)
                     ada2p1 (ACTIVE)
                     ada1p1 (ACTIVE)
mirror/p2  DEGRADED  ada0p2 (ACTIVE)
                     ada2p2 (ACTIVE)
mirror/p3  DEGRADED  ada0p3 (ACTIVE)
                     ada2p3 (ACTIVE)

p3 に ada0p3 を追加

TOP ↑ ↓

root@ns4:/home/hoge # gmirror forget p3
root@ns4:/home/hoge # gmirror status
     Name    Status  Components
mirror/p1  COMPLETE  ada0p1 (ACTIVE)
                     ada2p1 (ACTIVE)
                     ada1p1 (ACTIVE)
mirror/p2  DEGRADED  ada0p2 (ACTIVE)
                     ada2p2 (ACTIVE)
mirror/p3  COMPLETE  ada0p3 (ACTIVE)
                     ada2p3 (ACTIVE)
root@ns4:/home/hoge # gmirror insert p3 /dev/ada1p3
gmirror: Provider ada1p3 too small.

ディスクのサイズが小さく、p3 が追加できない。
元ディスクの p3 のサイズは 7381870
新ディスクの p3 のサイズは 7381864

メッセージ確認

> grep GEOM_MIRROR /var/log/messages
Aug 23 01:40:39 ns4 kernel: GEOM_MIRROR: Device p1: rebuilding provider ada1p1.
Aug 23 01:40:39 ns4 kernel: GEOM_MIRROR: Device p1: rebuilding provider ada1p1 finished.

p2 に ada2p2 を追加

TOP ↑ ↓

root@ns4:/home/hoge # gmirror status p2
    Name    Status  Components
mirror/p2  DEGRADED  ada0p2 (ACTIVE)
                     ada2p2 (ACTIVE)
root@ns4:/home/hoge # gmirror forget p2
root@ns4:/home/hoge # gmirror status p2
     Name    Status  Components
mirror/p2  COMPLETE  ada0p2 (ACTIVE)
                     ada2p2 (ACTIVE)
root@ns4:/home/hoge # gmirror insert p2 /dev/ada1p2
root@ns4:/home/hoge # gmirror status p2
     Name    Status  Components
mirror/p2  DEGRADED  ada0p2 (ACTIVE)
                     ada2p2 (ACTIVE)
                     ada1p2 (SYNCHRONIZING, 0%)

途中経過

root@ns4:/home/hoge # gmirror status p2

    Name    Status  Components

mirror/p2 DEGRADED ada0p2 (ACTIVE)

                    ada2p2 (ACTIVE)
                    ada1p2 (SYNCHRONIZING, 85%)

復旧完了状況確認

gmirror status

root@ns4:/home/hoge # gmirror status p2
     Name    Status  Components
mirror/p2  COMPLETE  ada0p2 (ACTIVE)
                     ada2p2 (ACTIVE)
                     ada1p2 (ACTIVE)

p2 が　DEGRADED から COMPLETE に変わった。

/var/log/messages

root@ns4:/home/hoge # grep GEOM_MIRROR /var/log/messages
Aug 23 01:45:20 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada1p2.
Aug 23 11:36:34 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada1p2 finished.

2018年5月は9時間で完了

May 28 18:32:14 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada1p2.
May 29 03:29:06 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada1p2 finished.

2019年5月は10時間で完了

May 23 01:57:29 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada0p2.
May 23 11:47:46 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada0p2 finished.

2020年8月は10時間で完了

Aug 23 01:45:20 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada1p2.
Aug 23 11:36:34 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada1p2 finished.

freebsd-swap 対応

TOP ↑ ↓

root@ns4:/home/hoge # swapinfo
Device          1K-blocks     Used    Avail Capacity
/dev/mirror/p3    3690932   198020  3492912     5%

root@ns4:/home/hoge # gmirror status p3
     Name    Status  Components
mirror/p3  COMPLETE  ada0p3 (ACTIVE)
                     ada2p3 (ACTIVE)

作業前

root@ns4:/home/hoge # gpart show ada1
=>        40  5860533088  ada1  GPT  (2.7T)
          40         128     1  freebsd-boot  (64K)
         168  5853151096     2  freebsd-ufs  (2.7T)
  5853151264     7381864     3  freebsd-swap  (3.5G)

freebsd-swap 削除

root@ns4:/home/hoge # gpart delete -i 3 ada1
ada1p3 deleted

root@ns4:/home/hoge # gpart show ada1
=>        40  5860533088  ada1  GPT  (2.7T)
          40         128     1  freebsd-boot  (64K)
         168  5853151096     2  freebsd-ufs  (2.7T)
  5853151264     7381864        - free -  (3.5G)

縮小して再作成

　gpart add -t freebsd-swap -s 7340032 ada1

root@ns4:/home/hoge # gpart add -t freebsd-swap -s 7340032 ada1
ada1p3 added

root@ns4:/home/hoge # gpart show ada1
=>        40  5860533088  ada1  GPT  (2.7T)
          40         128     1  freebsd-boot  (64K)
         168  5853151096     2  freebsd-ufs  (2.7T)
  5853151264     7340032     3  freebsd-swap  (3.5G)
  5860491296       41832        - free -  (20M)

swap 停止

root@ns4:/home/hoge # swapinfo
Device          1K-blocks     Used    Avail Capacity
/dev/mirror/p3    3690932   198020  3492912     5%

root@ns4:/home/hoge # swapoff /dev/mirror/p3

root@ns4:/home/hoge # swapinfo
Device          1K-blocks     Used    Avail Capacity

gmirror p3 削除

gmirror stop p3
gmirror destroy p3
gmirror label -v -b round-robin -s 65536 p3 ada1p3 ada0p3 ada2p3

root@ns4:/home/hoge # gmirror stop p3

root@ns4:/home/hoge # gmirror status
     Name    Status  Components
mirror/p1  COMPLETE  ada0p1 (ACTIVE)
                     ada2p1 (ACTIVE)
                     ada1p1 (ACTIVE)
mirror/p2  COMPLETE  ada0p2 (ACTIVE)
                     ada2p2 (ACTIVE)
                     ada1p2 (ACTIVE)
mirror/p3  COMPLETE  gptid/468c041d-f3c7-11e3-a841-0022cfe54d22 (ACTIVE)
                     gptid/46597fa0-7cb1-11e9-8d65-6805ca0d3e8b (ACTIVE)

root@ns4:/home/hoge # gmirror destroy p3

root@ns4:/home/hoge # gmirror status
     Name    Status  Components
mirror/p1  COMPLETE  ada0p1 (ACTIVE)
                     ada2p1 (ACTIVE)
                     ada1p1 (ACTIVE)
mirror/p2  COMPLETE  ada0p2 (ACTIVE)
                     ada2p2 (ACTIVE)
                     ada1p2 (ACTIVE)

root@ns4:/home/hoge # gmirror label -v -b round-robin -s 65536 p3 ada1p3 ada0p3 ada2p3
Metadata value stored on ada1p3.
Metadata value stored on ada0p3.
Metadata value stored on ada2p3.
Done.
root@ns4:/home/hoge # gmirror status
     Name    Status  Components
mirror/p1  COMPLETE  ada0p1 (ACTIVE)
                     ada2p1 (ACTIVE)
                     ada1p1 (ACTIVE)
mirror/p2  COMPLETE  ada0p2 (ACTIVE)
                     ada2p2 (ACTIVE)
                     ada1p2 (ACTIVE)
mirror/p3  COMPLETE  ada1p3 (ACTIVE)
                     ada0p3 (ACTIVE)
                     ada2p3 (ACTIVE)

root@ns4:/home/hoge # swapinfo
Device          1K-blocks     Used    Avail Capacity
root@ns4:/home/hoge # swapon /dev/mirror/p3
root@ns4:/home/hoge # swapinfo
Device          1K-blocks     Used    Avail Capacity
/dev/mirror/p3    3670012        0  3670012     0%

gmirror p3 縮小して再作成

起動確認

TOP ↑ ↓

3台のHDDを入れ替えて3台とも起動できることを確認する。
pmbr は正常か? bootcode は正常か? 確認する。

shutdown
HDD1-->HDD2-->HDD3-->HDD1
起動確認
shutdown
HDD1-->HDD2-->HDD3-->HDD1
起動確認
shutdown
HDD1-->HDD2-->HDD3-->HDD1
起動確認