RAID Controller Card Adaptation Guide
Key Modules for RAID Controller Card Adaptation: vpd, pcie_device, and storage
PCIe Card Discovery Using the .sr File of vpd
The standard topology is: root > EXU > BCU > Riser > PCIe. You must first configure the Connector.
For example, for the S920X20 device, start from the root.sr file and locate Connector_EXU_1. IdentifyMode is set to 3, which indicates that the system retrieves the next-level node from Tianchi. Run the busctl --user tree bmc.kepler.hwdiscovery | cat command to query Connector_EXU_1_01. Continue by running busctl --user introspect bmc.kepler.hwdiscovery /bmc/kepler/Connector/Connector_EXU_1_01 | cat to obtain the expansion board information.
~ ~ $ busctl --user introspect bmc.kepler.hwdiscovery /bmc/kepler/Connector/Connector_EXU_1_01 | cat
NAME TYPE SIGNATURE RESULT/VALUE FLAGS
bmc.kepler.Connector interface - - -
.Reload method a{ss}sssy - -
.AuxId property s "" emits-change writable
.Bom property s "14100513" emits-change
.Buses property as 33 "I2c_1" "I2c_2" "I2c_3" "I2c_4" "I2c... emits-change
.ChassisId property s "1" emits-change
.GroupId property u 4 emits-change
.GroupPosition property s "0101" emits-change
.Id property s "00000001010302044492" emits-change writable
.IdentifyMode property y 3 emits-change
.LoadStatus property y 0 emits-change
.ManagerId property s "1" emits-change
.Presence property y 1 emits-change writable
.SilkText property s "J6023" emits-change
.Slot property y 1 emits-change writable
.SystemId property y 1 emits-change
.Type property s "ExpandBoard" emits-changeAnalysis of 14100513_00000001010302044492.sr:Connector_BCU_1 is identified. Its IdentifyMode is set to 3, which indicates that the system retrieves the next-level node from Tianchi to obtain the basic board information.
~ ~ $ busctl --user introspect bmc.kepler.hwdiscovery /bmc/kepler/Connector/Connector_BCU_1_0101 | cat
NAME TYPE SIGNATURE RESULT/VALUE FLAGS
bmc.kepler.Connector interface - - -
.Reload method a{ss}sssy - -
.AuxId property s "" emits-change writable
.Bom property s "14060876" emits-change
.Buses property as 26 "I2c_1" "I2c_2" "I2c_8" "JtagMux_Jta... emits-change
.ChassisId property s "1" emits-change
.GroupId property u 8 emits-change
.GroupPosition property s "010101" emits-change
.Id property s "00000001020302031825" emits-change writable
.IdentifyMode property y 3 emits-change
.LoadStatus property y 0 emits-change
.ManagerId property s "1" emits-change
.Presence property y 1 emits-change writable
.SilkText property s "BCU" emits-change
.Slot property y 1 emits-change writable
.SystemId property y 1 emits-change
.Type property s "CPUBoard" emits-changeAnalysis of 14060876_00000001020302031825.sr: Multiple PCIeRiserCard objects are visible. IdentifyMode remains 3, indicating that Tianchi components are still being loaded. This is the basic board object, which contains several RiserCard objects. The following uses Connector_A2a as an example.
"Connector_A2a": {
"Bom": "14100513",
"Slot": 1,
"Position": 3,
"Presence": "<=/Scanner_A2a.Value",
"Id": "",
"AuxId": "",
"Buses": [
"Hisport_18"
],
"SystemId": 1,
"SilkText": "CpuBoard${Slot}",
"IdentifyMode": 3,
"Type": "PCIeRiserCard"
},The Buses property is Hisport_18, which can be seen as a specific I2C bus. The PCIeRiserCard object is obtained after the query.
~ ~ $ busctl --user introspect bmc.kepler.hwdiscovery /bmc/kepler/Connector/Connector_A2a_010101 | cat
NAME TYPE SIGNATURE RESULT/VALUE FLAGS
bmc.kepler.Connector interface - - -
.Reload method a{ss}sssy - -
.AuxId property s "" emits-change writable
.Bom property s "14100513" emits-change
.Buses property as 1 "Hisport_18" emits-change
.ChassisId property s "" emits-change
.GroupId property u 49 emits-change
.GroupPosition property s "01010103" emits-change
.Id property s "00000001040302044498" emits-change writable
.IdentifyMode property y 3 emits-change
.LoadStatus property y 0 emits-change
.ManagerId property s "1" emits-change
.Presence property y 1 emits-change writable
.SilkText property s "CpuBoard1" emits-change
.Slot property y 1 emits-change writable
.SystemId property y 1 emits-change
.Type property s "PCIeRiserCard" emits-changePCIe Device Update and RAID Controller Card Discovery Using pcie_device
1. Establishing Service Topology
The pcie_device component uses the CSR files configured within the vpd module to implement dynamic acquisition of PCIe device links. It provides PCIe device slot information to the BIOS. At this stage, the RiserCard has been analyzed. This module handles the analysis and loading of CSRs, which is considered core code and does not need to be modified.
Analysis of 14100513_00000001040302044498.sr: This is a single RiserCard object. Each RiserCard object can connect to multiple PCIe cards. Therefore, there is a Pca9545 object. The buses passed in is Hisport_18, which is divided into multiple I2C buses by Pca9545.
"I2cMux_Pca9545_PCA9545_1": {
"Connectors": [
"Connector_PCIE_SLOT1"
]
},
"I2cMux_Pca9545_PCA9545_2": {
"Connectors": [
"Connector_PCIE_SLOT2"
]
},
"I2cMux_Pca9545_PCA9545_3": {
"Connectors": [
"Connector_PCIE_SLOT3"
]
}For the current Connector_PCIE_SLOT3 object, IdentifyMode is 2, indicating that it is no longer a Tianchi component. Below it is the PCIe card. To determine which CSR the card should load, the system needs to retrieve the Id and AuxId properties. For a RAID controller card, these properties are obtained from the BIOS.
"Connector_PCIE_SLOT3": {
"Bom": "14140130",
"Slot": 3,
"Position": 3,
"Presence": 0,
"Buses": [
"I2cMux_Pca9545_PCA9545_3"
],
"SystemId": 1,
"SilkText": "RiserCard${Slot}",
"IdentifyMode": 2,
"Container": "Component_RiserCard",
"Type": "PCIe"
},2. Loading the PCIe Device and Retrieving and Setting Id and AuxId
After the service topology is established, the BIOS retrieves the PCIe device slot and CPU resource configuration information from the BMC. Then, the BIOS reports the device Bus/Device/Function (BDF) information of the PCIe devices to the BMC.
The BMC uses the device BDF information to send a request to the PMU. It queries the PCIe device quadruple information, which enables the loading of the corresponding CSR for the device.
In the code, the parse_pcie_card_bdf_data function parses the BDF information. It then calls the task_load_unload_device function to load the PCIe device, set the Id and AuxId property values, and set the Presence value to 1.
The load_unload_device function produces the following log output during loading:
device_loader.lua(214): [BizTopoLoader] Load PCIeCard, Slot=3, path=/bmc/kepler/Connector/Connector_PCIE_SLOT3_01010103, Id-AuxId=100010e2-10004010Based on the solution for managing standard PCIe devices through BIOS-reported BDF, the queried Connector properties include Id, AuxId, and Presence.
~ ~ $ busctl --user introspect bmc.kepler.hwdiscovery /bmc/kepler/Connector/Connector_PCIE_SLOT3_01010103 | cat
NAME TYPE SIGNATURE RESULT/VALUE FLAGS
bmc.kepler.Connector interface - - -
.Reload method a{ss}sssy - -
.AuxId property s "10004010" emits-change writable
.Bom property s "14140130" emits-change
.Buses property as 1 "I2cMux_Pca9545_PCA9545_3_01010103" emits-change
.ChassisId property s "" emits-change
.GroupId property u 57 emits-change
.GroupPosition property s "0101010303" emits-change
.Id property s "100010e2" emits-change writable
.IdentifyMode property y 2 emits-change
.LoadStatus property y 0 emits-change
.ManagerId property s "1" emits-change
.Presence property y 1 emits-change writable
.SilkText property s "RiserCard1" emits-change
.Slot property y 3 emits-change writable
.SystemId property y 1 emits-change
.Type property s "PCIe" emits-changeCorrect completion indicates that the CSR corresponding to the RAID controller card has been correctly identified and successfully loaded. The app.log file contains the following information:
[BizTopoLoader] Load PCIeCard, Slot=3, path=/bmc/kepler/Connector/Connector_PCIE_SLOT3_01010103, Id-AuxId=100010e2-10004010The framework.log file contains the following information:
hwdiscovery NOTICE: hwcomponent.lua(309): [self-discovery] name: Connector_PCIE_SLOT3_01010103, position: 0101010303, current: 1, previous: 0,uptime: 125 s
hwdiscovery NOTICE: init.lua(162): position: 0101010303, get csr data from /opt/bmc/sr/14140130_100010e2_10004010.sr, format version: 3.00, data version: 3.00
hwdiscovery NOTICE: hwcomponent.lua(205): position: 0101010303, load sr data successfully, uptime: 125 s, cost: 20ms
hwdiscovery NOTICE: hwcomponent.lua(226): position: 0101010303, start to process sr data, source: /opt/bmc/sr/14140130_100010e2_10004010.sr, format version: 3.00, data version: 3.00, uptime: 125 s
hwdiscovery NOTICE: hwcomponent.lua(309): [self-discovery] name: Connector_PCIE_SLOT1_01010103, position: 0101010301, current: 1, previous: 0,uptime: 125 sRAID Controller Card Initialization and Function Implementation Using storage
After the pcie_device module identifies the corresponding RAID controller card CSR, it calls the storage module to retrieve RAID controller card information.
The source code can be found at storage.
1. Initializing the RAID Controller Card
When the system detects a power-on event, it triggers the corresponding function, resulting in the following output:
storage NOTICE: bus_monitor_service.lua(84): [monitor-power] set power state from ONING to ON
storage NOTICE: controller_object.lua(150): controller init obj.Id = 255, object_id = 1
storage NOTICE: controller_object.lua(698): Controller_0, RefChip.Path:/bmc/kepler/Chip/Complex/Chip_RaidChip_0101010303
storage NOTICE: controller_object.lua(334): ctrl0 add_controller_to_link_topo successfully
storage NOTICE: controller_object.lua(336): ctrl0 add_controller_to_sml successfullyIf the configuration is incorrect, the following occurs:
storage NOTICE: init.lua(534): sml: set i2c chip, ctrl_idx=0, chip=/bmc/kepler/Chip/Complex/Chip_RaidChip_0101010303
storage NOTICE: controller_object.lua(698): Controller_0, RefChip.Path:/bmc/kepler/Chip/Complex/Chip_RaidChip_0101010303
storage NOTICE: controller_object.lua(334): ctrl0 add_controller_to_link_topo successfully
storage ERROR: tasks.lua(83): task [Controller.register_controller.0] error: ...bmc/apps/storage/lualib/controller/controller_object.lua:723: [Storage] Failed to add controller 0, ret: 4357You can also see the corresponding error messages in the framework.log file:
framework NOTICE: l_sml_adapter.cpp(240): register_sml_adapter_function: controller type id is 14
framework ERROR: l_sml_adapter.cpp(159): register_sml_adapter_fun_table: g_sml_adapter is already generate
framework ERROR: l_sml_adapter.cpp(181): register_pd_log_parse_table: g_pd_log_adapter is already generate
framework ERROR: l_sml_adapter.cpp(71): register_pd_log_parse: g_pd_log_parse is already generate
framework ERROR: adapter.c(603): Failed to load lsi sml library /usr/lib64/libsml_lsi.so for MegaRAID SAS Controller. error : /usr/lib64/libsml_lsi.so: cannot open shared object file: No such file or directory
framework ERROR: adapter.c(918): smlib : Add controller management [Ctrl index 0, Ctrl ID 0] failed, return 0x11d8To resolve this, add the following configuration under libmgmt_protocol in the manifest.yaml file:
- conan: libmgmt_protocol
options:
storelib_enable: trueAfter adding the configuration, recompile and verify to ensure that the process works correctly. You can then query the relevant D-Bus information to obtain RAID controller card information.
~ ~ $ busctl --user introspect bmc.kepler.storage /bmc/kepler/Systems/1/Storage/Controllers/Controller_1_0101010303 | cat
NAME TYPE SIGNATURE RESULT/VALUE FLAGS
bmc.kepler.Inventory.Hardware interface - - -
.AssetName property s "MegaRAID 9560-8i 4GB" emits-change
.AssetTag property s "N/A" emits-change
.AssetType property s "PCIe RAID Card" emits-change
.FirmwareVersion property s "5.290.02-3997" emits-change
.ManufactureDate property s "N/A" emits-change
.Manufacturer property s "Broadcom" emits-change
.Model property s "SAS3908" emits-change
.PCBVersion property s "N/A" emits-change
.PartNumber property s "06030622" emits-change
.SerialNumber property s "SPD3511231" emits-change
.Slot property s "0" emits-change
.UUID property s "N/A" emits-change2. Obtaining RAID Controller Card Information
After initialization is complete, the system begins updating the RAID controller card information by calling c_controller:start() as follows:
self:start_update_task()
self:start_update_pd_list_task()
self:start_update_ld_list_task()
self:start_update_phy_err_task()These tasks are responsible for periodically obtaining RAID controller card information, such as the controller, logical drives, physical drives, and physical drive errors. Example: start_update_task calls the get_ctrl_info function. Through the mapping between Lua and C code, this eventually calls the C function in the library for the corresponding card.
After information is obtained, the on_update:on function triggers a data refresh.
self:update_static_controller_info(info)
self:update_controller_info(info)
self:update_asset_data_info()These update functions update data into the properties of their own objects, which makes the information available on the D-Bus.
3. Setting RAID Controller Card Information
The entry point for RAID controller card configuration commands is the rpc_service_controller file. Pay attention to the interfaces defined in this file. Example: ctrl_task_operate is an entry point that executes as an asynchronous task to avoid blocking. Example: The ClearForeignConfig interface executes directly rather than asynchronously.
Key Considerations for RAID Controller Card Configuration
Method of Finding the Corresponding CSR
Check whether the vpd module already supports the corresponding RAID controller card. Obtain the quadruple information for the RAID controller card, such as: manufacturer ID (0x1000), device ID (0x10e2), sub-manufacturer ID (0x1000), and sub-device ID (0x4010). The corresponding CSR file is 14140130_100010e2_10004010.sr.
Current Low-level Communication Only Supports I2C and MCTP over PCIe Modes
Additional development is required if you need to add other modes.
Broadcom RAID controller cards use I2C for low-level communication with an I2C address of 0x02. If you use i2cdetect, it responds at address 0x01. The latest HBA card (9600) uses the Management Component Transport Protocol (MCTP), which has not yet been adapted. PMC boards use MCTP over PCIe. You must first establish the MCTP link correctly before data updates can proceed. You can check if the link is normal by running busctl --user tree bmc.kepler.mctpd | cat. Huawei cards use MCTP over PCIe for low-level communication but also provide several I2C addresses for separately querying information such as Lm75 and eeprom.