*FreeBSD-11 でディスク障害が発生した。 *FreeBSD gmirror GPT 3T HDD トリプルミラーでの障害 *セクタエラー *障害復旧手順の記録 *zfs ではない *GPT のバックアップ領域と gmirror の管理領域が重複するため、GPTではディスク丸ごと gmirror できない。 *FreeBSD GPT gmirror 失敗 *MBR だったらディスク丸ごと gmirror ができる。 *回避のため、GPTで複数のパーティションを作成し、それぞれのパーティションで gmirror を動作している。 *複数の gmirror が動作している。 *ここでは、bootcode + freebsd-swap + freebsd-ufs !!!発生状況 !4/4 ""SMART error (FailedOpenDevice) detected on host: ns4 This message was generated by the smartd daemon running on: host name: ns4 DNS domain: nona.to The following warning/error was logged by the smartd daemon: Device: /dev/ada0, unable to open ATA device Device info: TOSHIBA DT01ACA300, S/N:144K58KGS, WWN:5-000039-ff4d5e59e, FW:MX6OABB0, 3.00 TB !5/4 ""SMART error (FailedOpenDevice) detected on host: ns4 This message was generated by the smartd daemon running on: host name: ns4 DNS domain: nona.to The following warning/error was logged by the smartd daemon: Device: /dev/ada1, unable to open ATA device Device info: TOSHIBA DT01ACA300, S/N:Z7647YTAS, WWN:5-000039-fe6de1e60, FW:MX6OABB0, 3.00 TB *I/Oエラー root@ns4:/home/hoge # time dd if=/dev/ada1 of=/dev/null bs=1048576 skip=1295380 dd: /dev/ada1: Input/output error 5+0 records in 5+0 records out 5242880 bytes transferred in 14.679095 secs (357166 bytes/sec) 0.000u 0.001s 0:14.68 0.0% 0+0k 43+0io 0pf+0w *1.29GB 付近でIOエラー !過去の対応 *FreeBSD gmirror 障害復旧手順例 201805 *FreeBSD gmirror 障害復旧手順例 201905 !!!作業概要 *gmirror ada0 ada1 ada2 のトリプルミラー *ada1 が故障 *ada1 を入れ替え *ada1 のパーティション設定 *ada1 の起動コード書き込み *トリプルミラー再開 !FreeBSD 正常時状況 */boot/loader.conf geom_mirror_load="YES" */etc/fstab # Device Mountpoint FStype Options Dump Pass# /dev/mirror/p2 / ufs rw 1 1 /dev/mirror/p3 none swap sw 0 0 *gmirror 正常時 root@ns4:/home/hoge # gmirror status -s mirror/p1 COMPLETE ada0p1 (ACTIVE) mirror/p1 COMPLETE ada2p1 (ACTIVE) mirror/p1 COMPLETE ada1p1 (ACTIVE) mirror/p2 COMPLETE ada0p2 (ACTIVE) mirror/p2 COMPLETE ada2p2 (ACTIVE) mirror/p2 COMPLETE ada1p2 (ACTIVE) mirror/p3 COMPLETE ada0p3 (ACTIVE) mirror/p3 COMPLETE ada2p3 (ACTIVE) mirror/p3 COMPLETE ada1p3 (ACTIVE) !!!作業記録 !!状況確認 *パーティション情報 root@ns4:/home/hoge # gpart list Geom name: ada0 modified: false state: OK fwheads: 16 fwsectors: 63 last: 5860533134 first: 34 entries: 128 scheme: GPT Providers: 1. Name: ada0p1 Mediasize: 65536 (64K) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r1w1e1 efimedia: HD(1,GPT,eaa06e43-7cb0-11e9-8d65-6805ca0d3e8b,0x28,0x80) rawuuid: eaa06e43-7cb0-11e9-8d65-6805ca0d3e8b rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f label: (null) length: 65536 offset: 20480 type: freebsd-boot index: 1 end: 167 start: 40 2. Name: ada0p2 Mediasize: 2996813361152 (2.7T) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r1w1e1 efimedia: HD(2,GPT,35c63824-7cb1-11e9-8d65-6805ca0d3e8b,0xa8,0x15cdfff78) rawuuid: 35c63824-7cb1-11e9-8d65-6805ca0d3e8b rawtype: 516e7cb6-6ecf-11d6-8ff8-00022d09712b label: (null) length: 2996813361152 offset: 86016 type: freebsd-ufs index: 2 end: 5853151263 start: 168 3. Name: ada0p3 Mediasize: 3779517440 (3.5G) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r1w1e1 efimedia: HD(3,GPT,46597fa0-7cb1-11e9-8d65-6805ca0d3e8b,0x15ce00020,0x70a36e) rawuuid: 46597fa0-7cb1-11e9-8d65-6805ca0d3e8b rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b label: (null) length: 3779517440 offset: 2996813447168 type: freebsd-swap index: 3 end: 5860533133 start: 5853151264 Consumers: 1. Name: ada0 Mediasize: 3000592982016 (2.7T) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r3w3e6 Geom name: ada1 modified: false state: OK fwheads: 16 fwsectors: 63 last: 5860533134 first: 34 entries: 128 scheme: GPT Providers: 1. Name: ada1p1 Mediasize: 65536 (64K) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r1w1e1 efimedia: HD(1,GPT,afd1aa88-7d1d-11e9-8d65-6805ca0d3e8b,0x28,0x80) rawuuid: afd1aa88-7d1d-11e9-8d65-6805ca0d3e8b rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f label: (null) length: 65536 offset: 20480 type: freebsd-boot index: 1 end: 167 start: 40 2. Name: ada1p2 Mediasize: 2996813361152 (2.7T) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r1w1e1 efimedia: HD(2,GPT,55594ddf-7d1e-11e9-8d65-6805ca0d3e8b,0xa8,0x15cdfff78) rawuuid: 55594ddf-7d1e-11e9-8d65-6805ca0d3e8b rawtype: 516e7cb6-6ecf-11d6-8ff8-00022d09712b label: (null) length: 2996813361152 offset: 86016 type: freebsd-ufs index: 2 end: 5853151263 start: 168 3. Name: ada1p3 Mediasize: 3779517440 (3.5G) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r1w1e1 efimedia: HD(3,GPT,59e22286-7d1e-11e9-8d65-6805ca0d3e8b,0x15ce00020,0x70a36e) rawuuid: 59e22286-7d1e-11e9-8d65-6805ca0d3e8b rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b label: (null) length: 3779517440 offset: 2996813447168 type: freebsd-swap index: 3 end: 5860533133 start: 5853151264 Consumers: 1. Name: ada1 Mediasize: 3000592982016 (2.7T) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r3w3e6 Geom name: ada2 modified: false state: OK fwheads: 16 fwsectors: 63 last: 5860533134 first: 34 entries: 128 scheme: GPT Providers: 1. Name: ada2p1 Mediasize: 65536 (64K) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r1w1e1 efimedia: HD(1,GPT,461f6d0d-f3c7-11e3-a841-0022cfe54d22,0x28,0x80) rawuuid: 461f6d0d-f3c7-11e3-a841-0022cfe54d22 rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f label: (null) length: 65536 offset: 20480 type: freebsd-boot index: 1 end: 167 start: 40 2. Name: ada2p2 Mediasize: 2996813361152 (2.7T) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r1w1e1 efimedia: HD(2,GPT,461fe080-f3c7-11e3-a841-0022cfe54d22,0xa8,0x15cdfff78) rawuuid: 461fe080-f3c7-11e3-a841-0022cfe54d22 rawtype: 516e7cb6-6ecf-11d6-8ff8-00022d09712b label: (null) length: 2996813361152 offset: 86016 type: freebsd-ufs index: 2 end: 5853151263 start: 168 3. Name: ada2p3 Mediasize: 3779517440 (3.5G) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r1w1e1 efimedia: HD(3,GPT,468c041d-f3c7-11e3-a841-0022cfe54d22,0x15ce00020,0x70a36e) rawuuid: 468c041d-f3c7-11e3-a841-0022cfe54d22 rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b label: (null) length: 3779517440 offset: 2996813447168 type: freebsd-swap index: 3 end: 5860533133 start: 5853151264 Consumers: 1. Name: ada2 Mediasize: 3000592982016 (2.7T) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r3w3e6 *gmirror status 確認 root@ns4:/home/hoge # gmirror status Name Status Components mirror/p1 COMPLETE ada0p1 (ACTIVE) ada1p1 (ACTIVE) ada2p1 (ACTIVE) mirror/p2 COMPLETE ada0p2 (ACTIVE) ada1p2 (ACTIVE) ada2p2 (ACTIVE) mirror/p3 COMPLETE ada0p3 (ACTIVE) ada1p3 (ACTIVE) ada2p3 (ACTIVE) *正常に見えるが、ada1 でIOエラー root@ns4:/home/hoge # time dd if=/dev/ada1 of=/dev/null bs=1048576 skip=1295380 dd: /dev/ada1: Input/output error 5+0 records in 5+0 records out 5242880 bytes transferred in 14.679095 secs (357166 bytes/sec) 0.000u 0.001s 0:14.68 0.0% 0+0k 43+0io 0pf+0w *ada1 1.29GB 付近でIOエラー !GPT パーティション情報の確認 *ada0 ada2 確認 gpart show ada0 gpart show ada1 gpart show ada2 root@ns4:/home/hoge # gpart show ada0 => 34 5860533101 ada0 GPT (2.7T) 34 6 - free - (3.0K) 40 128 1 freebsd-boot (64K) 168 5853151096 2 freebsd-ufs (2.7T) 5853151264 7381870 3 freebsd-swap (3.5G) 5860533134 1 - free - (512B) root@ns4:/home/hoge # gpart show ada1 => 34 5860533101 ada1 GPT (2.7T) 34 6 - free - (3.0K) 40 128 1 freebsd-boot (64K) 168 5853151096 2 freebsd-ufs (2.7T) 5853151264 7381870 3 freebsd-swap (3.5G) 5860533134 1 - free - (512B) root@ns4:/home/hoge # gpart show ada2 => 34 5860533101 ada2 GPT (2.7T) 34 6 - free - (3.0K) 40 128 1 freebsd-boot (64K) 168 5853151096 2 freebsd-ufs (2.7T) 5853151264 7381870 3 freebsd-swap (3.5G) 5860533134 1 - free - (512B) !!ada1 交換 !物理ディスク ada1 除去 */var/log/messages Aug 23 00:36:45 ns4 kernel: ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 Aug 23 00:36:45 ns4 kernel: ada1: s/n 74S3K1H4FSDA detached Aug 23 00:36:45 ns4 kernel: GEOM_MIRROR: Device p3: provider ada1p3 disconnected.GEOM_MIRROR Aug 23 00:36:45 ns4 kernel: : Device p1: provider ada1p1 disconnected. Aug 23 00:36:45 ns4 kernel: GEOM_MIRROR: Device p2: provider ada1p2 disconnected. Aug 23 00:36:45 ns4 kernel: (ada1:ahcich1:0:0:0): Periph destroyed root@ns4:/home/hoge # gmirror status Name Status Components mirror/p1 DEGRADED ada0p1 (ACTIVE) ada2p1 (ACTIVE) mirror/p2 DEGRADED ada0p2 (ACTIVE) ada2p2 (ACTIVE) mirror/p3 DEGRADED ada0p3 (ACTIVE) ada2p3 (ACTIVE) !物理ディスク ada1 接続 */var/log/messages Aug 23 00:40:24 ns4 kernel: ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 Aug 23 00:40:24 ns4 kernel: ada1: ATA8-ACS SATA 3.x device Aug 23 00:40:24 ns4 kernel: ada1: Serial Number 29O6V7GAS Aug 23 00:40:24 ns4 kernel: ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) Aug 23 00:40:24 ns4 kernel: ada1: Command Queueing enabled Aug 23 00:40:24 ns4 kernel: ada1: 2861588MB (5860533168 512 byte sectors) Aug 23 00:40:24 ns4 kernel: GEOM: ada1: the secondary GPT header is not in the last LBA. Aug 23 00:40:24 ns4 kernel: GEOM: diskid/DISK-29O6V7GAS: the secondary GPT header is not in the last LBA. !ada1 GPT パーティション作成 *ada1 GPT パーティションを確認 root@ns4:/home/hoge # gpart show ada1 => 34 5860533100 ada1 GPT (2.7T) [CORRUPT] 34 65501 - free - (32M) 65535 5860467375 1 linux-data (2.7T) 5860532910 224 - free - (112K) *Linux のパーティションが存在する。 gpart delete -i 1 ada1 root@ns4:/home/hoge # gpart delete -i 1 ada1 gpart: table 'ada1' is corrupt: Operation not permitted root@ns4:/home/hoge # gpart recover /dev/ada1 ada1 recovered root@ns4:/home/hoge # gpart show ada1 => 40 5860533088 ada1 GPT (2.7T) 40 65495 - free - (32M) 65535 5860467375 1 linux-data (2.7T) 5860532910 218 - free - (109K) root@ns4:/home/hoge # gpart delete -i 1 ada1 ada1p1 deleted root@ns4:/home/hoge # gpart show ada1 => 40 5860533088 ada1 GPT (2.7T) 40 5860533088 - free - (2.7T) *ada0 GPT 管理情報作成 gpart create -s gpt /dev/ada1 root@ns4:/home/hoge # gpart create -s gpt /dev/ada1 gpart: geom 'ada1': File exists *GPT 管理情報は既に存在している。 *ada1 GPT 初期状況確認 root@ns4:/home/hoge # gpart show ada1 => 40 5860533088 ada1 GPT (2.7T) 40 5860533088 - free - (2.7T) !ada0 FreeBSD pmbr インストール :pmbr:Proactive Master Boot Record root@ns4:/home/hoge # ls -l /boot/pmbr root@ns4:/home/hoge # gpart bootcode -b /boot/pmbr ada1 bootcode written to ada1 *MBR ツールによる破壊を防止するための疑似MBR情報+GPT起動コード !ada1 FreeBSD bootcode インストール *コマンド ls -l /boot/gptboot gpart add -b 40 -s 128 -t freebsd-boot ada1 gpart bootcode -p /boot/gptboot -i 1 ada1 *ada0 GPT 情報より freebsd-boot 領域のサイズは 128 *ログ root@ns4:/home/hoge # ls -l /boot/gptboot -r--r--r-- 1 root wheel 59074 Aug 10 2019 /boot/gptboot root@ns4:/home/hoge # gpart show ada1 => 40 5860533088 ada1 GPT (2.7T) 40 5860533088 - free - (2.7T) root@ns4:/home/hoge # gpart add -b 40 -s 128 -t freebsd-boot ada1 ada1p1 added root@ns4:/home/hoge # gpart show ada1 => 40 5860533088 ada1 GPT (2.7T) 40 128 1 freebsd-boot (64K) 168 5860532960 - free - (2.7T) *(参考) GPT パーティションの除去 *コマンド gpart delete -i 1 ada1 *ログ root@ns4:/home/hoge # gpart show ada1 => 34 5860533101 ada1 GPT (2.7T) 34 6 - free - (3.0K) 40 5860533088 1 freebsd-boot (2.7T) 5860533128 7 - free - (3.5K) root@ns4:/home/hoge # gpart delete -i 1 ada1 ada1p1 deleted root@ns4:/home/hoge # gpart show ada1 => 34 5860533101 ada1 GPT (2.7T) 34 5860533101 - free - (2.7T) !ada0 GPT パーティション作成 *コマンド gpart add -t freebsd-ufs -s 5853151096 ada1 gpart add -t freebsd-swap -s 7381870 -a 2 ada1 *ログ root@ns4:/home/hoge # gpart add -t freebsd-ufs -s 5853151096 ada1 ada1p2 added root@ns4:/home/hoge # gpart add -t freebsd-swap -s 7381870 -a 2 ada1 gpart: autofill: No space left on device root@ns4:/home/hoge # gpart show ada1 => 40 5860533088 ada1 GPT (2.7T) 40 128 1 freebsd-boot (64K) 168 5853151096 2 freebsd-ufs (2.7T) 5853151264 7381864 - free - (3.5G) root@ns4:/home/hoge # gpart add -t freebsd-swap -s 7381864 -a 2 ada1 ada1p3 added root@ns4:/home/hoge # gpart show ada1 => 40 5860533088 ada1 GPT (2.7T) 40 128 1 freebsd-boot (64K) 168 5853151096 2 freebsd-ufs (2.7T) 5853151264 7381864 3 freebsd-swap (3.5G) *ada2 と*ほぼ*同じパーティション情報を ada1 に作成した。 *freebsd-swap のサイズが小さいので復旧できていない。別途対応 *gpart backup と gpart restore でパーティション情報を復旧するべき(要確認) !GPT パーティション状況確認 gpart show ada0 gpart show ada2 *同じパーティション情報であることを確認した。 !gmirror 再構築 *単純に insert はできない。 root@ns4:/home/hoge # gmirror insert p1 /dev/ada1p1 gmirror: Not all disks connected. !DEGRADED　確認 root@ns4:/home/hoge # gmirror status Name Status Components mirror/p1 DEGRADED ada0p1 (ACTIVE) ada2p1 (ACTIVE) mirror/p2 DEGRADED ada0p2 (ACTIVE) ada2p2 (ACTIVE) mirror/p3 DEGRADED ada0p3 (ACTIVE) ada2p3 (ACTIVE) !p1 DEGRADED 状況を解消する root@ns4:/home/hoge # gmirror forget p1 root@ns4:/home/hoge # gmirror status Name Status Components mirror/p1 COMPLETE ada0p1 (ACTIVE) ada2p1 (ACTIVE) mirror/p2 DEGRADED ada0p2 (ACTIVE) ada2p2 (ACTIVE) mirror/p3 DEGRADED ada0p3 (ACTIVE) ada2p3 (ACTIVE) !p1 に ada1p1 を追加 root@ns4:/home/hoge # gmirror insert p1 /dev/ada1p1 root@ns4:/home/hoge # gmirror status Name Status Components mirror/p1 COMPLETE ada0p1 (ACTIVE) ada2p1 (ACTIVE) ada1p1 (ACTIVE) mirror/p2 DEGRADED ada0p2 (ACTIVE) ada2p2 (ACTIVE) mirror/p3 DEGRADED ada0p3 (ACTIVE) ada2p3 (ACTIVE) !p3 に ada0p3 を追加 root@ns4:/home/hoge # gmirror forget p3 root@ns4:/home/hoge # gmirror status Name Status Components mirror/p1 COMPLETE ada0p1 (ACTIVE) ada2p1 (ACTIVE) ada1p1 (ACTIVE) mirror/p2 DEGRADED ada0p2 (ACTIVE) ada2p2 (ACTIVE) mirror/p3 COMPLETE ada0p3 (ACTIVE) ada2p3 (ACTIVE) root@ns4:/home/hoge # gmirror insert p3 /dev/ada1p3 gmirror: Provider ada1p3 too small. *ディスクのサイズが小さく、p3 が追加できない。 *元ディスクの p3 のサイズは 7381870 *新ディスクの p3 のサイズは 7381864 *メッセージ確認 > grep GEOM_MIRROR /var/log/messages Aug 23 01:40:39 ns4 kernel: GEOM_MIRROR: Device p1: rebuilding provider ada1p1. Aug 23 01:40:39 ns4 kernel: GEOM_MIRROR: Device p1: rebuilding provider ada1p1 finished. !p2 に ada2p2 を追加 root@ns4:/home/hoge # gmirror status p2 Name Status Components mirror/p2 DEGRADED ada0p2 (ACTIVE) ada2p2 (ACTIVE) root@ns4:/home/hoge # gmirror forget p2 root@ns4:/home/hoge # gmirror status p2 Name Status Components mirror/p2 COMPLETE ada0p2 (ACTIVE) ada2p2 (ACTIVE) root@ns4:/home/hoge # gmirror insert p2 /dev/ada1p2 root@ns4:/home/hoge # gmirror status p2 Name Status Components mirror/p2 DEGRADED ada0p2 (ACTIVE) ada2p2 (ACTIVE) ada1p2 (SYNCHRONIZING, 0%) *途中経過 root@ns4:/home/hoge # gmirror status p2 Name Status Components mirror/p2 DEGRADED ada0p2 (ACTIVE) ada2p2 (ACTIVE) ada1p2 (SYNCHRONIZING, 85%) *復旧完了状況確認 gmirror status root@ns4:/home/hoge # gmirror status p2 Name Status Components mirror/p2 COMPLETE ada0p2 (ACTIVE) ada2p2 (ACTIVE) ada1p2 (ACTIVE) *p2 が　DEGRADED から COMPLETE に変わった。 */var/log/messages root@ns4:/home/hoge # grep GEOM_MIRROR /var/log/messages Aug 23 01:45:20 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada1p2. Aug 23 11:36:34 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada1p2 finished. *2018年5月は9時間で完了 May 28 18:32:14 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada1p2. May 29 03:29:06 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada1p2 finished. *2019年5月は10時間で完了 May 23 01:57:29 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada0p2. May 23 11:47:46 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada0p2 finished. *2020年8月は10時間で完了 Aug 23 01:45:20 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada1p2. Aug 23 11:36:34 ns4 kernel: GEOM_MIRROR: Device p2: rebuilding provider ada1p2 finished. !freebsd-swap 対応 root@ns4:/home/hoge # swapinfo Device 1K-blocks Used Avail Capacity /dev/mirror/p3 3690932 198020 3492912 5% root@ns4:/home/hoge # gmirror status p3 Name Status Components mirror/p3 COMPLETE ada0p3 (ACTIVE) ada2p3 (ACTIVE) *作業前 root@ns4:/home/hoge # gpart show ada1 => 40 5860533088 ada1 GPT (2.7T) 40 128 1 freebsd-boot (64K) 168 5853151096 2 freebsd-ufs (2.7T) 5853151264 7381864 3 freebsd-swap (3.5G) *freebsd-swap 削除 root@ns4:/home/hoge # gpart delete -i 3 ada1 ada1p3 deleted root@ns4:/home/hoge # gpart show ada1 => 40 5860533088 ada1 GPT (2.7T) 40 128 1 freebsd-boot (64K) 168 5853151096 2 freebsd-ufs (2.7T) 5853151264 7381864 - free - (3.5G) *縮小して再作成 　gpart add -t freebsd-swap -s 7340032 ada1 root@ns4:/home/hoge # gpart add -t freebsd-swap -s 7340032 ada1 ada1p3 added root@ns4:/home/hoge # gpart show ada1 => 40 5860533088 ada1 GPT (2.7T) 40 128 1 freebsd-boot (64K) 168 5853151096 2 freebsd-ufs (2.7T) 5853151264 7340032 3 freebsd-swap (3.5G) 5860491296 41832 - free - (20M) *swap 停止 root@ns4:/home/hoge # swapinfo Device 1K-blocks Used Avail Capacity /dev/mirror/p3 3690932 198020 3492912 5% root@ns4:/home/hoge # swapoff /dev/mirror/p3 root@ns4:/home/hoge # swapinfo Device 1K-blocks Used Avail Capacity *gmirror p3 削除 gmirror stop p3 gmirror destroy p3 gmirror label -v -b round-robin -s 65536 p3 ada1p3 ada0p3 ada2p3 root@ns4:/home/hoge # gmirror stop p3 root@ns4:/home/hoge # gmirror status Name Status Components mirror/p1 COMPLETE ada0p1 (ACTIVE) ada2p1 (ACTIVE) ada1p1 (ACTIVE) mirror/p2 COMPLETE ada0p2 (ACTIVE) ada2p2 (ACTIVE) ada1p2 (ACTIVE) mirror/p3 COMPLETE gptid/468c041d-f3c7-11e3-a841-0022cfe54d22 (ACTIVE) gptid/46597fa0-7cb1-11e9-8d65-6805ca0d3e8b (ACTIVE) root@ns4:/home/hoge # gmirror destroy p3 root@ns4:/home/hoge # gmirror status Name Status Components mirror/p1 COMPLETE ada0p1 (ACTIVE) ada2p1 (ACTIVE) ada1p1 (ACTIVE) mirror/p2 COMPLETE ada0p2 (ACTIVE) ada2p2 (ACTIVE) ada1p2 (ACTIVE) root@ns4:/home/hoge # gmirror label -v -b round-robin -s 65536 p3 ada1p3 ada0p3 ada2p3 Metadata value stored on ada1p3. Metadata value stored on ada0p3. Metadata value stored on ada2p3. Done. root@ns4:/home/hoge # gmirror status Name Status Components mirror/p1 COMPLETE ada0p1 (ACTIVE) ada2p1 (ACTIVE) ada1p1 (ACTIVE) mirror/p2 COMPLETE ada0p2 (ACTIVE) ada2p2 (ACTIVE) ada1p2 (ACTIVE) mirror/p3 COMPLETE ada1p3 (ACTIVE) ada0p3 (ACTIVE) ada2p3 (ACTIVE) root@ns4:/home/hoge # swapinfo Device 1K-blocks Used Avail Capacity root@ns4:/home/hoge # swapon /dev/mirror/p3 root@ns4:/home/hoge # swapinfo Device 1K-blocks Used Avail Capacity /dev/mirror/p3 3670012 0 3670012 0% *gmirror p3 縮小して再作成 !起動確認 *3台のHDDを入れ替えて3台とも起動できることを確認する。 *pmbr は正常か? bootcode は正常か? 確認する。 *shutdown *HDD1-->HDD2-->HDD3-->HDD1 *起動確認 *shutdown *HDD1-->HDD2-->HDD3-->HDD1 *起動確認 *shutdown *HDD1-->HDD2-->HDD3-->HDD1 *起動確認