mctpd is an independent MCTP route management service offered by openUBMC. It provides resource collaboration interfaces for MCTP-related communication.

Function Overview ​

Management Component Transport Protocol (MCTP) is a physical media–independent protocol for information exchange among components in a computer system. This protocol is independent of the underlying physical bus and is a "data link layer" protocol independent of the bus.

MCTP is defined by the Distributed Management Task Force Platform Management Component Intercommunications (DMTF PMCI) Working Group. This protocol provides interfaces to access the monitoring and management functions of the platform management subsystem, so that manageable functions of the platform can be accessed, transmitted, and configured through the DMTF Common Information Model (CIM).

mctpd Startup Process ​

After the mctpd is started, it listens Status of the Pmu object of the compute component and SmBiosStatus of the SmBios object of the BIOS component. If either signal changes to 1 (indicating the server is powered on) while mctpd is in power-off state, the mctpd is changed to the power-on state, the MCTP driver is reset, and the MCTP self-discovery process is started.

MCTP self-discovery process:

1. The IMU sends the Prepare for Endpoint Discovery request to the BMC.
2. The BMC processes the Prepare for Endpoint Discovery request, sets the Owner's EID and physical address, and returns a response.
3. The IMU sends the Endpoint Discovery request to the BMC.
4. The BMC processes the Endpoint Discovery request and returns a response.
5. The IMU sends the Set Endpoint ID request to the BMC.
6. The BMC processes the Set Endpoint ID request, sets the BMC's EID and physical address, and returns a response.
7. The mctpd starts a task to obtain and update the routing table from the IMU every 5 seconds.

The service component calls the mctp_lib.get_endpoint_and_transport method to create a Transport and obtain the Endpoint object (if Routing exists), and sends a request by using the Send or Request method of the Endpoint object.

MCTP over PCIe Request Message Processing ​

MCTP over SMBus/SMBus_OEM Request Message Processing ​

Endpoint Object ​

The mctpd can be loaded by multiple components. The PCIe Endpoint depends on the MCTP bus owner and MCTP driver to ensure the transmission implementation. Users need to create the Endpoint of the MessageType they are using and use the methods encapsulated in the EndPoint for MCTP communication.

The SMBus Endpoint depends on the CSR to configure the MctpBinding object, Endpoint object, and associated Chip object.

The SMBus_OEM Endpoint depends on the CSR to configure the MctpBinding object, Endpoint object, and associated Chip object, and depends on the link supported by specific hardware.

CSR Configuration Example ​

//CSR configuration example of SMBus Endpoints
"MctpBinding_1": {          //MctpBinding object configuration
    "BmcSMBusEid": 8,       //BMC EID used by the SMBus protocol
    "BmcSMBusPhyAddr": 16   //BMC physical address used by the SMBus protocol
}

"Endpoint_1": {                             //Endpoint object configuration
    EID of "TargetEid": "${Slot} |> expr(8 + $1)", //Endpoint EID, usually the EID of MctpBinding plus the slot number.
    "TargetPhyAddr": 73,                    //Endpoint physical address (7 bits)
    "MessageType": 2,                       //Endpoint message protocol type. For details about the supported types, see the table "Endpoints That Can Be Created".
    "MediumType": 128,                      ///Endpoint physical layer protocol type. For details about the supported types, see the table "Endpoints That Can Be Created".
    "RefChip": "#/Chip_SmbusChip"           //Chip object associated with the Endpoint
}

"Chip_SmbusChip": {     //Chip object configuration
    "Address": 58,      //Address of the physical hardware corresponding to the Endpoint object
    "AddrWidth": 1,     //Hardware address bit width
    "OffsetWidth": 1,   //Hardware offset bit width
    "WriteTmout": 100,  //Write timeout
    "ReadTmout": 100,   //Read timeout
    "HealthStatus": 0   //Health status
}

//Configure the Chip under the corresponding bus in ManagementTopology, declaring under which bus the chip is located.
"I2cMux_9545Chan": {
    "Chips": [
        "Chip_SmbusChip"
    ]
}

Using Endpoint Objects for MCTP Communication ​

Endpoint offers the Request and Send methods for components to perform MCTP communication. The Request method request a return value, while the Send method does not.

Currently, NC-SI over MCTP, NVMe over MCTP, and PLDM over MCTP are encapsulated in the libmgmt_protocol component. You are advised to use the encapsulated methods for communication.

Example of NC-SI Endpoints ​

-- context indicates the user context.
-- ncsi_req_ctx is the protocol header information.
-- ncsi_rsp_ctx is the verification item for the protocol header. Currently, only PacketType needs to be verified.
-- request_data is a specific command parameter (binary string).
-- timeout indicates the timeout interval for waiting for a response. The unit is ms.

-- Return value required:
local ok, rsp_data_bin = pcall(endpoint.Request, endpoint, context, request_data_bin, timeout, ncsi_req_ctx, ncsi_rsp_ctx)
-- No return value required:
local ok = pcall(endpoint.Send, endpoint, context, request_data_bin, timeout, ncsi_req_ctx)

Example of NVMe Endpoints ​

-- context indicates the user context.
-- nvme_req_ctx is the protocol header information.
-- nvme_rsp_ctx is the verification item for the protocol header. Currently, only MsgType needs to be verified.
-- request_data is a specific command parameter (binary string).
-- timeout indicates the timeout interval for waiting for a response. The unit is ms.

-- Return value required:
local ok, rsp_data_bin = pcall(endpoint.Request, endpoint, context, request_data_bin, timeout, nvme_req_ctx, nvme_rsp_ctx)
-- No return value required:
local ok = pcall(endpoint.Send, endpoint, context, request_data_bin, timeout, nvme_req_ctx)

Example of PLDM Endpoints ​

-- context indicates the user context.
-- pldm_req_ctx is the protocol header information.
-- pldm_rsp_ctx is the verification item for the protocol header. Currently, only CommandCode needs to be verified.
-- request_data is a specific command parameter (binary string).
-- timeout indicates the timeout interval for waiting for a response. The unit is ms.

-- Return value required:
local ok, rsp_data_bin = pcall(endpoint.Request, endpoint, context, request_data_bin, timeout, pldm_req_ctx, pldm_rsp_ctx)
-- No return value required:
local ok = pcall(endpoint.Send, endpoint, context, request_data_bin, timeout, pldm_req_ctx)

Example of Vendor Defined-PCI Endpoints ​

-- context indicates the user context.
-- request_data contains the vendor-defined protocol header information and a specific command parameter (binary string).
-- timeout indicates the timeout interval for waiting for a response. The unit is ms.

-- Return value required:
local ok, rsp_data_bin = pcall(endpoint.Request, endpoint, context, request_data_bin, timeout, {}, {})
-- No return value required:
local ok = pcall(endpoint.Send, endpoint, context, request_data_bin, timeout, {})

Endpoints That Can Be Created ​

For other upper-layer protocols, you can create common MCTP Endpoints to encapsulate them.

mctp_lib Usage Guide ​

To facilitate the creation of MCTP Endpoint, the mctpd provides the mctp_lib module for developers to quickly create Endpoint and Transport objects.

How to Use ​

local mctp_lib = require 'mctp_lib'

Obtaining the MCTP Port Information of the BMC ​

-- Return value struct:
-- {
--   OwnerEid:         EID of the IMU
--   OwnerPhyAddr:     PhyAddr of the IMU
--   BmcEid:            BMC EID
--   BmcPhyAddr:       PhyAddr of the BMC
--   DiscoverFinished: whether Endpoint Discovery has been completed
--   RoutingTableReady: whether the routing table is obtained
--  }
-- If the server is powered off, no value is returned.
-- To use this function, you need to reference bmc.kepler.Systems.Mctp.MctpBinding.PCIe in service.json.

local obj = mctp_lib.get_mctp_pcie_binding(bus)

Obtaining the PCIe Device's phy_addr ​

-- Calculate the device's phy_addr based on the BDF information of the device.
-- Example: bdf:(0x82, 0, 0) => phy_addr:0x0082, bdf:(0x6, 0, 0) => phy_addr:0x0006

local phy_addr = mctp_lib.bdf_to_phy_addr(self.Bus, self.Device, self.Function)

Creating or Obtaining the Endpoint and Transport Objects in the Resource Tree Based on the Module Name, Physical Address, Protocol Type, and Position ​

-- MODULE_NAME indicates the module name, and its value is a string.
-- msg_type indicates the protocol type. The value can be the currently supported MessageTypeCode.
-- For the PCIe Endpoint, Position does not need to be passed. For the SMBus or SMBus_OEM Endpoint, GroupPosition in the sr file where the Endpoint is located needs to be passed.
-- To use this function, you need to reference bmc.kepler.Systems.Mctp.PCIeTransport and bmc.kepler.Systems.Mctp.PCIeEndpoint in service.json.

local ok, endpoint, transport = pcall(mctp_lib.get_endpoint_and_transport, bus, MODULE_NAME, phy_addr, msg_type, position)

Obtaining the PCIe Endpoint Object in the Resource Tree Based on the Physical Address and Protocol Type ​

-- msg_type indicates the protocol type. The value can be the currently supported MessageTypeCode.
-- If the Endpoint object is not added to the tree, the function fails.
-- To use this function, you need to reference bmc.kepler.Systems.Mctp.PCIeEndpoint in service.json.

local ok, endpoint = pcall(mctp_lib.get_pcie_endpoint, bus, phy_addr, msg_type)

Obtaining the SMBus Endpoint Object in the Resource Tree Based on the Physical Address, Protocol Type, and Position ​

-- msg_type indicates the protocol type. The value can be the currently supported MessageTypeCode.
-- position indicates GroupPosition in the sr file where the SMBus Endpoint is located.
-- If the Endpoint object is not added to the tree, the function fails.
-- To use this function, you need to reference bmc.kepler.Systems.Mctp.PCIeEndpoint in service.json.

local ok, endpoint = pcall(mctp_lib.get_smbus_endpoint, bus, position, phy_addr, msg_type)

FAQs ​

How Do I Distinguish the SMBus Endpoints from the PCIe Endpoints? ​

You can determine the Endpoint type based on the resource tree path. The PCIe Endpoint's path is /bmc/kepler/Systems/:SystemId/Mctp/Endpoint/phyaddr/msg_type, and the physical address is unique. The SMBus configuration comes from the CSR configuration, which may result in multiple Endpoint sharing the same physical address. Therefore, Position needs to be added as the unique identifier to distinguish different Endpoint. The path of the SMBus Endpoint is thus /bmc/kepler/Systems/:SystemId/Mctp/Endpoint/position_phyaddr/msg_type.

├── /bmc/kepler/Systems
│   └── /bmc/kepler/Systems/1
│       └── /bmc/kepler/Systems/1/Mctp
│           └── /bmc/kepler/Systems/1/Mctp/Endpoint
│               ├── /bmc/kepler/Systems/1/Mctp/Endpoint/0101020E_29
│               │   └── /bmc/kepler/Systems/1/Mctp/Endpoint/0101020E_29/4
│               ├── /bmc/kepler/Systems/1/Mctp/Endpoint/0101020F_29
│               │   └── /bmc/kepler/Systems/1/Mctp/Endpoint/0101020F_29/4
│               └── /bmc/kepler/Systems/1/Mctp/Endpoint/171
│                   └── /bmc/kepler/Systems/1/Mctp/Endpoint/171/4

Why Cannot MCTP Packets Be Sent (over PCIe VDM)? ​

This issue involves many aspects. You are advised to perform the following steps to locate the fault:

1. Check that the IMU is powered on. ​

You can check whether the power supply is on by visiting the web page or viewing the mctpd log for the keyword Power ON.

2025-04-26 18:07:16.430174 mctpd NOTICE: mctp_mdb_mgmt.lua(246): mctp_mdb_mgmt: pmu status change to 1, OS Power ON

2. Check whether the powered-on IMU sends a discovery packet to the BMC. ​

The mctpd can send and receive packets only after being connected to the IMU. After the IMU is powered on, it broadcasts the discovery packet. After receiving the packet, the mctpd replies with a message. Then, the IMU sets up a connection with the BMC. You can check which step has been reached through the logs. The keyword set owner_eid indicates the step of receiving the discovery packet and setting the EID and physical address of the IMU), set bmc_eid indicates the step of receiving the set_ep_id packet and setting the EID and physical address of the BMC, start update route table indicates the step of starting to obtain the routing table from the IMU, and add routing entry indicates the step of receiving the routing table packet returned by the IMU and setting the routing table.)

2025-04-26 18:07:17.038622 mctpd NOTICE: mctp_engine.lua(272): mctp_engine: set owner_eid=8, owner_phy_addr=8
2025-04-26 18:07:17.109490 mctpd NOTICE: mctp_engine.lua(281): mctp_engine: set bmc_eid=11, bmc_phy_addr=768
2025-04-26 18:07:17.109862 mctpd NOTICE: mctp_discovery.lua(234): mctp_discovery: start update route table
2025-04-26 18:07:17.335796 mctpd NOTICE: mctp_routing_table.lua(52): mctp_routing_table: add routing entry, phys_address=150, starting_eid=9, entry_type=0

After receiving the routing table, the mctpd creates the Routing object. The resource tree path is /bmc/kepler/Systems/1/Mctp/Routing/phyaddr. If the routing table does not contain the expected NIC and disk addresses (phyaddr), check whether the corresponding hardware responds to the discovery packet of the IMU.

└─/bmc/kepler/Systems/1/Mctp/Routing
  ├─/bmc/kepler/Systems/1/Mctp/Routing/150
  ├─/bmc/kepler/Systems/1/Mctp/Routing/173
  └─/bmc/kepler/Systems/1/Mctp/Routing/3

3. After the IMU is powered on, verify that the calling component has registered the Transport. ​

After the IMU is powered on, verify that the calling component has registered the Transport and the Transport address is in the routing table: Components such as network_adapter and storage will register Transport objects through the interface provided by the mctpd. The mctpd determines whether to create an Endpoint object based on whether the physical address of the Transport is in the routing table. If there is no routing table of the corresponding address, the mctpd does not create an Endpoint object. The path of the Transport is /bmc/kepler/Systems/1/Mctp/Transport/phyaddr/msg_type.

    ├─/bmc/kepler/Systems
    │ └─/bmc/kepler/Systems/1
    │   ├─/bmc/kepler/Systems/1/Mctp
    │   │ └─/bmc/kepler/Systems/1/Mctp/Transport
    │   │   └─/bmc/kepler/Systems/1/Mctp/Transport/150
    │   │     ├─/bmc/kepler/Systems/1/Mctp/Transport/150/1
    │   │     └─/bmc/kepler/Systems/1/Mctp/Transport/150/2

Log keywords for Endpoint creation are add eid info and create endpoint. You can identify which component is registering an Endpoint by module_name. Information such as the corresponding physical address is also recorded in logs.

2025-04-26 18:08:01.820337 mctpd NOTICE: mctp_eid_table.lua(81): mctp_eid_table: add eid info, module_name=network_adapter,msg_src_eid=9,phy_addr=150,msg_type=1, medium_type=15, ep_status=1
2025-04-26 18:08:01.822045 mctpd NOTICE: init.lua(86): mctp_endpoints: create endpoint, phy_addr=150, eid=9, msg_type=1, medium_type=15
2025-04-26 18:08:01.980190 mctpd NOTICE: mctp_eid_table.lua(81): mctp_eid_table: add eid info, module_name=network_adapter,msg_src_eid=9,phy_addr=150,msg_type=2, medium_type=15, ep_status=1
2025-04-26 18:08:01.986384 mctpd NOTICE: init.lua(86): mctp_endpoints: create endpoint, phy_addr=150, eid=9, msg_type=2, medium_type=15

4. If the preceding points are checked (the power supply is on, the routing table is available, and the Endpoint exists) but the packet still cannot be sent, check at which part of the link the message is dropped. ​

The message link of the PCIe Endpoint is BMC > mctp driver > IMU > Endpoint. To clarify that the issue is not caused by the BMC, add log printing before the BMC sends packets to the driver for verification.

5. The protocol code is incorrect. ​

If both received and sent packets are printed, but the BMC still reports a request timeout, analyze the logs in detail to determine if there are structural issues in the response packet, which cause the packet to fail validation and be silently discarded by the mctpd.

Common fault locating log keywords are as follows:

• nvme_protocol: unmatch crc
• ncsi_protocol: check sum
• ncsi_protocol: incorrect payload size
• mctp_protocol: unexpected fragment size
• mctp_protocol: sequence number
• sm_bus_protocol: incorrect packet size
• sm_bus_protocol: header error, wr
• sm_bus_protocol: incorrect payload size, data.size
• pcie_vdm_protocol: vdm header error, vdm_code
• pcie_vdm_protocol: incorrect vdm header, payload_len
• pcie_vdm_protocol: incorrect payload size
• pcie_vdm_protocol: incorrect packet size