To reiterate my problem: I had a misconfigured mirror metadevice under Solaris 8/DiskSuite - the /opt filesystem was mounted on a raw partition making up the mirror (c1t0d0s5) instead of the metadevice (d5) itself. I had tried simply unmounting /opt and remounting it on the mirror device, but that resulted in a corrupted directory structure; doing a subsequent metadetach & metattach did not help. My setup was: metadevice d5 was a mirror, consisting of two submirrors d15 (c1t0d0s5) and d25 (c1t0d1s5). As it turns out, I was correct in the commands I needed but wrong about the sequence. What I did successfully was to (1) detach submirror d25, leaving only the single submirror d15 (which was the disk slice I had been using); (2) unmount /opt, remount onto the mirror d5; and (3) attach d25 back onto d5. After a few minutes of resyncing, all looks OK. Now, I have to do this for /usr. My plan is to (1) edit vfstab so that /usr points to the mirror, (2) detach the second submirror, (3) reboot, (4) reattach the second submirror. After, of course, a verified backup. I also have to do this for swap. Thanks to: Ken Rossman Wesley W. Garland Peter Junker Thomas M. Payerle Alex Stade Darren Dunham In addition, I found this http://www.blacksheepnetworks.com/security/resources/labs/disksuite.html - which summarizes disksuite usage well. And this - http://www.physics.umd.edu/rgroups/pnce/pcs-docs/Glue/disksuite/disksuite-boot-mirror.html - which calls out some warnings about what I did wrong in the first place (not detaching mirrors before remounting onto the mirror). steve edberg ======================= = Summary of replies: = ======================= The thing to do here is to break the mirror (metadetach), then reattach it (metattach). You have to decide which side of the mirror is more correct in this case, and leave that submirror attached to the mirror metadevice, while detaching the other submirror. But I'd make the assumption in this case that it was the physical device that *was* mounted directly that you'd consider correct in this case. --------------------------------------------------------------- This should do the trick: metareplace -e d5 c1t1d0s5 then "metastat | grep %" -- if it says "Resyncing, 0% done" you're golden. PS: Backup first!! --------------------------------------------------------------- How to get issues with volumes under Solstice Diksmanager fixed lookup up the documentation. If you don't have it, the lookit up under http://docs.sun.com Dealing with the file systems /, /usr, /opt and swap needs special steps, otherwise there is going to be file system corruption. --------------------------------------------------------------- I thinkyour bestbet is to get all problematic filesystems "unmirrored", mounted using the physical device, then metadetach all submirrors but the one corresponding to the physical device you were using, then metaclear the mirror metadevice. Then just "remirror"; make a submirror metadevice for the physical device in use (this may be redundant, as probably still around) (if you do make it, use the -f option as it should be already mounted). Create the mirror metadevice with only a single submirror (the one corresponding to the mounted physicsal device). Now you must remount the filesystem so the mirror metadevice is the mount point. Then you can attach other submirrors. --------------------------------------------------------------- I'll explain briefly what I believe has happened, and then tell you my conclusion. When you first do a; metainit -f d15 1 1 c1t0d0s5 metainit d5 -m d15 You're telling SDS/SVM that you've create a mirror and defined which drive to mirror _from_. When you then; metainit d25 1 1 c1t1d0s5 metattach d5 d25 SDS/SVM knows which device to sync from. However, once sync is completed, things change. Unless you specified -g and/or -r when you created the mirror (from metainit(1M)); The following read options for mirrors are supported: -g Enables the geometric read option, which results in faster performance on sequential reads. -r Directs all reads to the first submirror. This should only be used when the devices comprising the first submirror are substantially faster than those of the second mirror. This flag can- not be used with the -g flag. You'll end up with the default, which is a round-robin style read (from metainit(1M)); If neither the -g nor -r flags are specified, reads are made in a round-robin order from all submirrors in the mirror. This enables load balancing across the submir- rors. So. My theory is that you're reading in a round-robin fashion, where the data on d25 is considerably older than the data on d15. As for what you can do, I would (since you have a good backup) attempt to detach d25 from d5 and re-attach it; metadetach d5 d25 metattach d5 d25 That should sync up the mirror from d15. --------------------------------------------------------------- <SNIP> > My question is: is there an easy way to refresh a metadevice from a > physical slice so that I can simply remount onto the mirror device? Yes. metadetach the side that's not mounted, remount onto the mirror, then metaattach the component back. If you do this later, then corruption will be rewritten onto both sides of the mirror.. *ouch*. I > know I could simply do a newfs on the d5 metadevice and restore the > backup, but I'd like to avoid that (especially since the /usr > partition is misconfigured this way as well). I assume pointing swap > at a metadevice will be no problem by editing /etc/vfstab & rebooting > will not be a problem since there's nothing persistent on swap, but > perhaps I'm missing something. The most you could do at this point is to resync to one side (sounds like you've done that) and fsck. But that will only make everything consistent, not necessarily "right". The problem is that the corruption has already occured, and probably been rewritten onto the disk.... </SNIP> ===================== = Original message: = ===================== Hi - I have a Sparc/Solaris 8 system with some mirror metadevices set up using Solaris Disksuite. However, some of the filesystems were mounted via /etc/vfstab onto one of the physical disk slices that made up the mirror, rather than the mirror metadevice. To remedy this, I first tested by unmounting the /opt filesystem (because my /opt was small, I had a current verified backup and there was nothing critical there), and remounting it onto the mirror device. In summary: /opt originally mounted on /dev/dsk/c1t0d0s5 c1t0d0s5 one part of mirror d5; metastat -i d5 reports all ok umount /opt mount -F ufs /dev/md/dsk/d5 /opt Now, I took a quick look at /opt and things *seemed* ok. However, problems soon became evident; eg, ls and ls -l reported different files/directories, trying to cd into some subdirectories of /opt gave me an IO error, recent files disappeared. I remounted /opt back onto the physical partition c1t0d0s5, and everything was OK again. It was as if the directory structure of the metadevice had not been updated, so anything (directories & files) that was older than approximately a year was visible. I am currently unable to trace things back to *why* the metadevice directory was updated then; it was over a year since the metadevices had been created, and much older than the last reboot; it *may* have been about the same time that the filesystems last needed fscking (thank <deity> for UFS logging!). I had checked metastat -i, and mirror were reported all ok, so I assumed that things would be fine. I was wrong. I tried again doing a metadetach and metattach in hopes that would resync the mirror's directory, that didn't help either. My question is: is there an easy way to refresh a metadevice from a physical slice so that I can simply remount onto the mirror device? I know I could simply do a newfs on the d5 metadevice and restore the backup, but I'd like to avoid that (especially since the /usr partition is misconfigured this way as well). I assume pointing swap at a metadevice will be no problem by editing /etc/vfstab & rebooting will not be a problem since there's nothing persistent on swap, but perhaps I'm missing something. Thanks in advance; will summarize - steve edberg Some details: uname -a ======== SunOS pgfjupiter 5.8 Generic_117000-03 sun4u sparc SUNW,Sun-Fire-280R Excerpt from current /etc/vfstab: ================================= # device device mount FS fsck mount mount # to mount to fsck point type pass at boot options ################# ################## ############ ##### # ### ############# /dev/dsk/c1t0d0s1 - - swap - no - /dev/md/dsk/d0 /dev/md/rdsk/d0 / ufs 1 no logging /dev/dsk/c1t0d0s6 /dev/rdsk/c1t0d0s6 /usr ufs 1 no logging /dev/dsk/c1t0d0s5 /dev/rdsk/c1t0d0s5 /opt ufs 2 yes logging metastat -p d5 ============== d5 -m d15 d25 1 d15 1 1 c1t0d0s5 d25 1 1 c1t1d0s5 (swap, /, /opt and /usr were all intended to be mounted on mirrors d1, d0, d5 & d6 respectively; only / is actually currently running on the mirror metadevice). metastat -i =========== d0: Mirror Submirror 0: d10 State: Okay Submirror 1: d20 State: Okay Pass: 1 Read option: roundrobin (default) Write option: parallel (default) Size: 8389656 blocks d10: Submirror of d0 State: Okay Size: 8389656 blocks Stripe 0: Device Start Block Dbase State Hot Spare c1t0d0s0 0 No Okay d20: Submirror of d0 State: Okay Size: 8389656 blocks Stripe 0: Device Start Block Dbase State Hot Spare c1t1d0s0 0 No Okay d1: Mirror Submirror 0: d11 State: Okay Submirror 1: d21 State: Okay Pass: 1 Read option: roundrobin (default) Write option: parallel (default) Size: 1048707 blocks d11: Submirror of d1 State: Okay Size: 1048707 blocks Stripe 0: Device Start Block Dbase State Hot Spare c1t0d0s1 0 No Okay d21: Submirror of d1 State: Okay Size: 1048707 blocks Stripe 0: Device Start Block Dbase State Hot Spare c1t1d0s1 0 No Okay d3: Mirror Submirror 0: d13 State: Okay Submirror 1: d23 State: Okay Pass: 1 Read option: roundrobin (default) Write option: parallel (default) Size: 2889 blocks d13: Submirror of d3 State: Okay Size: 2889 blocks Stripe 0: Device Start Block Dbase State Hot Spare c1t0d0s3 2889 Yes Okay d23: Submirror of d3 State: Okay Size: 2889 blocks Stripe 0: Device Start Block Dbase State Hot Spare c1t1d0s3 2889 Yes Okay d4: Mirror Submirror 0: d14 State: Okay Submirror 1: d24 State: Okay Pass: 1 Read option: roundrobin (default) Write option: parallel (default) Size: 2889 blocks d14: Submirror of d4 State: Okay Size: 2889 blocks Stripe 0: Device Start Block Dbase State Hot Spare c1t0d0s4 2889 Yes Okay d24: Submirror of d4 State: Okay Size: 2889 blocks Stripe 0: Device Start Block Dbase State Hot Spare c1t1d0s4 2889 Yes Okay d5: Mirror Submirror 0: d15 State: Okay Submirror 1: d25 State: Okay Pass: 1 Read option: roundrobin (default) Write option: parallel (default) Size: 8389656 blocks d15: Submirror of d5 State: Okay Size: 8389656 blocks Stripe 0: Device Start Block Dbase State Hot Spare c1t0d0s5 0 No Okay d25: Submirror of d5 State: Okay Size: 8389656 blocks Stripe 0: Device Start Block Dbase State Hot Spare c1t1d0s5 0 No Okay <SNIP> -- +--------------- my people are the people of the dessert, ---------------+ | Steve Edberg http://pgfsun.ucdavis.edu/ | | UC Davis Genome Center sbedberg@ucdavis.edu | | Bioinformatics programming/database/sysadmin (530)754-9127 | +---------------- said t e lawrence, picking up his fork ----------------+ _______________________________________________ sunmanagers mailing list sunmanagers@sunmanagers.org http://www.sunmanagers.org/mailman/listinfo/sunmanagersReceived on Mon Jun 6 09:23:32 2005
This archive was generated by hypermail 2.1.8 : Thu Mar 03 2016 - 06:43:47 EST