[[TOC(depth=-1)]] = LSI MegaRAID SAS = [[Image(wiki:LSI:logo_lsi.gif)]] [[BR]][[BR]] = 1. Card information = MegaRAID SAS is the current high-end RAID controllers series by LSI. [[BR]]It is fully hardware RAIDs controllers supporting RAID5, at least, with SAS or SATA interfaces. [[BR]]If you're a looking for information about MegaRAID SCSI connectors, please look at [wiki:LSIMegaRAID] instead. All theses card can be used with stock Linux kernel which includes a '''working driver'''. [[BR]]It's quite new and thus, may be missing in some not-up-to-date distributions. There is currently '''no known opensource tool''' for theses cards. [[BR]]Some old MegaRAID SAS can be used with megactl, but none of current cards works. [[BR]]However LSI provide ''megacli'', a '''proprietary management command line utility''' which is rather '''hard to use'''. [[BR]] = 2. Linux kernel drivers = || Driver || Supported cards || || megaraid_sas || LSI MegaRAID SAS || ''megaraid_sas is part of mainstream Linux kernel and should be available in all current distributions. [[BR]]However, please that most of old distributions won't have this driver. If your card use ''megaraid_mm'' or ''megaraid_mbox'' driver, please look at [wiki:LSIMegaRAID] instead. Some ''lspci -nn'' output examples: * 02:0e.0 RAID bus controller ![0104]: Dell !PowerEdge Expandable RAID controller 5 ![1028:0015] * 01:00.0 RAID bus controller ![0104]: LSI Logic / Symbios Logic MegaRAID SAS 1078 (rev 04) ![1000:0060] * 04:0e.0 RAID bus controller ![0104]: LSI Logic / Symbios Logic MegaRAID SAS ![1000:0411] [[BR]] = 3. Management and reporting tools = '''megactl''' includes a SAS compliant binary named '''megasasctl'''. It seems to '''work on old card''' but fails with the new one. [[BR]] If megasasctl doesn't work for you, you will have to use the '''proprietary cli utility''' from LSI: '''megaclisas'''. For managing the card there are no alternatives to ''megaclisas''. == 3.1. megactl == Despites megasasctl doesn't seem to work with recent cards, you should really give it a try. === 3.1.1. Quickstart and output example === Print current controller status: {{{ server:~# megasasctl a0 PERC 5/i Integrated encl:1 ldrv:1 batt:good a0d0 136GiB RAID 1 1x2 optimal a0e8s0 136GiB a0d0 online a0e8s1 136GiB a0d0 online }}} {{{ [root@server ~]# megasasctl a0 PERC 5/i Integrated encl:1 ldrv:2 batt:good a0d0 67GiB RAID 1 1x2 optimal a0d1 836GiB RAID 5 1x4 optimal a0e8s0 68GiB a0d0 online a0e8s1 68GiB a0d0 online a0e8s2 279GiB a0d1 online a0e8s3 279GiB a0d1 online a0e8s4 279GiB a0d1 online a0e8s5 279GiB a0d1 online }}} There are several switches which are interresting: * '''-H''': Only print lines which are not ok.[[BR]]If nothing is printer, everything is fine * '''-B''': Ignore batttery problems when running -H.[[BR]]In fact megasasctl can't define if your controller has a battery or not.[[BR]]If you don't have one, use this parameter. === 3.1.2. Periodic checks === You can write your own script around megasasctl to check your adapter status health periodically. However, I already did this for you. See megaraid-status below. == 3.2. megaraid-status == === 3.2.1. About megaraid-status === '''megaraidsas-status''' is a wrapper script around megactl with periodics checks. [[BR]]It is available in the packages repository too. [[BR]]The packages comes with a python wrapper around megasasctl and an initscript that '''periodic''' run this wrapper to '''check status'''. [[BR]]It keeps a file with latest status and thus is able to detect RAID '''status changes and/or brokeness'''. [[BR]]It will log a ligne to '''syslog''' when something failed and will send you a '''mail'''. [[BR]]Until arrays are healthy again a '''reminder''' will be sent each 2 hours. === 3.2.2. Wrapper output example === {{{ server:~# megaraidsas-status -- Arrays informations -- -- ID | Type | Size | Status a0d0 | RAID 1 | 136GiB | optimal -- Disks informations -- ID | Model | Status | Warnings a0e8s0 | SEAGATE ST3146854SS 136GiB | online a0e8s1 | SEAGATE ST3146854SS 136GiB | online }}} {{{ [root@server ~]# megaraidsas-status -- Arrays informations -- -- ID | Type | Size | Status a0d0 | RAID 1 | 67GiB | optimal a0d1 | RAID 5 | 836GiB | optimal -- Disks informations -- ID | Model | Status | Warnings a0e8s0 | FUJITSU MBA3073RC 68GiB | online a0e8s1 | FUJITSU MBA3073RC 68GiB | online a0e8s2 | SEAGATE ST3300656SS 279GiB | online a0e8s3 | SEAGATE ST3300656SS 279GiB | online a0e8s4 | SEAGATE ST3300656SS 279GiB | online a0e8s5 | SEAGATE ST3300656SS 279GiB | online }}} == 3.3. megacli == === 3.3.1. About megacli === '''megacli''' is a proprietary tool by LSI which can perform both '''reporting and management''' for MegaRAID SAS cards. [[BR]]However it's really hard to use because it's use '''tones of command line parameters''' and there's '''no documentation'''. === 3.3.2. Quickstart and output example === Get all adapters status and config: {{{ server:~# megacli -AdpAllInfo -aAll Adapter #0 ============================================================================== Versions ================ Product Name : PERC 5/i Integrated Serial No : 12345 FW Package Build: 5.2.1-0067 Mfg. Data ================ Mfg. Date : 00/00/00 Rework Date : 00/00/00 Revision No : @��A Battery FRU : N/A Image Versions In Flash: ================ Boot Block Version : R.2.3.12 BIOS Version : MT28-8 MPT Version : MPTFW-00.10.61.00-IT FW Version : 1.03.40-0316 WebBIOS Version : 1.03-04 Ctrl-R Version : 1.04-019A [...] }}} Logical drive 0 on adapter 0 status and type: {{{ server:~# megacli -LDInfo -L0 -a0 Adapter 0 -- Virtual Drive Information: Virtual Disk: 0 (Target Id: 0) Name:raid1 RAID Level: Primary-1, Secondary-0, RAID Level Qualifier-0 Size:237824MB State: Optimal Stripe Size: 64kB Number Of Drives:2 Span Depth:1 Default Cache Policy: WriteBack, ReadAheadNone, Direct, No Write Cache if Bad BBU Current Cache Policy: WriteBack, ReadAheadNone, Direct, No Write Cache if Bad BBU Access Policy: Read/Write Disk Cache Policy: Disk's Default Exit Code: 0x00 }}} Display, disable or enable automatic rebuild on adapter 0: {{{ server:~# megacli -AdpAutoRbld -Dsply -a0 Adapter 0: AutoRebuild is Enabled. Exit Code: 0x00 server:~# megacli -AdpAutoRbld -Dsbl -a0 Adapter 0: AutoRebuild is Disabled. Exit Code: 0x00 server:~# megacli -AdpAutoRbld -Enbl -a0 Adapter 0: AutoRebuild is Enabled. Exit Code: 0x00 }}} Get and modify rebuild rate: {{{ server:~# megacli -AdpGetProp RebuildRate -a0 Adapter 0: Rebuild Rate = 30% Exit Code: 0x00 server:~# megacli -AdpSetProp RebuildRate 60 -a0 Adapter 0: Set rebuild rate to 60% success. Exit Code: 0x00 }}} Show physical disks from first controller: {{{ server:~# megacli -PDList -a0 [...] Enclosure Device ID: 32 Slot Number: 1 Device Id: 1 Sequence Number: 9 Media Error Count: 0 Other Error Count: 0 Predictive Failure Count: 0 Last Predictive Failure Event Seq Number: 0 PD Type: SAS Raw Size: 140014MB [0x11177328 Sectors] Non Coerced Size: 139502MB [0x11077328 Sectors] Coerced Size: 139392MB [0x11040000 Sectors] Firmware state: Rebuild SAS Address(0): 0x5000c5000c8579d1 SAS Address(1): 0x0 Connected Port Number: 1(path0) Inquiry Data: SEAGATE ST3146855SS S5283LN6CNGM Foreign State: None }}} We can see that disk 32,1 (enclosure id = 32, slot = 1) is currently rebuilding (firmware state). Let's check this operation progress: {{{ server:~# megacli -PDRbld -ShowProg -PhysDrv [32:1] -aALL Rebuild Progress on Device at Enclosure 32, Slot 1 Completed 51% in 10 Minutes. }}} ==== Create a RAID6 array with megacli ==== === 3.3.3. Periodic checks === You can write your own script around megacli to check your adapter status health periodically. However, I already did this for you. See megaclisas-status below. == 3.4. megaclisas-status == === 3.4.1. About megaclisas-status === '''megaclisas-status''' is a wrapper script around '''megacli''' that report summarized RAID status with periodic checks feature. [[BR]]It is available in the packages repository too. The packages comes with a python wrapper around megacli and an initscript that '''periodic''' run this wrapper to '''check status'''. [[BR]]It keeps a file with latest status and thus is able to detect RAID '''status changes and/or brokeness'''. [[BR]]It will log a ligne to '''syslog''' when something failed and will send you a '''mail'''. [[BR]]Until arrays are healthy again a '''reminder''' will be sent each 2 hours. === 3.4.2. Wrapper output example === {{{ server:~# megaclisas-status -- Controller informations -- -- ID | Model c0 | PERC 5/i Integrated -- Arrays informations -- -- ID | Type | Size | Status | InProgress c0u0 | RAID1 | 238G | Optimal | None -- Disks informations -- ID | Model | Status c0u0p0 | ATA ST3250620NS 3BKH 5QE4PW70 | Online c0u0p1 | ATA WDC WD2500YS-18S6C07 WD-WCANY3450500 | Online }}} Another example: {{{ server:~# megaclisas-status -- Controller informations -- -- ID | Model c0 | PERC 6/i Integrated -- Arrays informations -- -- ID | Type | Size | Status | InProgress c0u0 | RAID5 | 279G | Optimal | Background Initialization: Completed 42%, Taken 46 min. -- Disks informations -- ID | Model | Status c0u0p0 | SEAGATE ST3146855SS S5283LN6CLF9 | Online c0u0p1 | SEAGATE ST3146855SS S5283LN6CNGM | Online c0u0p2 | SEAGATE ST3146855SS S5283LN6CGW4 | Online }}} The same example but with second disque rebuilding: {{{ server:~# megaclisas-status -- Controller informations -- -- ID | Model c0 | PERC 6/i Integrated -- Arrays informations -- -- ID | Type | Size | Status | InProgress c0u0 | RAID5 | 279G | Degraded | None -- Disks informations -- ID | Model | Status c0u0p0 | SEAGATE ST3146855SS S5283LN6CLF9 | Online c0u0p1 | SEAGATE ST3146855SS S5283LN6CNGM | Rebuild c0u0p2 | SEAGATE ST3146855SS S5283LN6CGW4 | Online There is at least one disk/array in a NOT OPTIMAL state. }}} == 3.5. About /dev/megaraid_sas_ioctl_node == All theses tools requires this device node to be created. [[BR]]It has to be done by hand. Proprietary tools creates the device node at startup. [[BR]]I made some wrappers around binaries from megactl package to create the node if it doesn't exist yet. [[BR]] = 4. BIOS upgrade from a Linux system = Dell cards can be flashed using firmware-tools. [[BR]]See http://linux.dell.com/wiki/index.php/Repository/firmware for more informations. [[BR]]However this will only work on !RedHat, CentOS, SuSE and Fedora. Even Ubuntu is listed in the wikipage, LSI card upgrade is not supported. [[BR]]We use a Fedora 8 nfsroot booted by PXE to update our Dell's firmware running Debian. We don't have any informations yet about upgrading non-Dell cards. [[BR]][[BR]]