Background
Unlike regular service code, rackmount, as a configuration file used for northbound interface parsing, has the following limitations in its log capabilities:
- All configuration elements except plugin does not have the log capability.
- Adding logs to plugins conflicts with the lightweight design of the configuration file and does not help with fault locating.
For certain faults that occur during development and maintenance, specific strategies must be adopted to locate them.
Fault Locating Using Debug Logs
openUBMC supports the use of debug logs for fault demarcation and locating through some interfaces. The specific usage is as follows:
Redfish/web_backend Interface
Enable debug logs:
mdbctl
attack redfish
dloglevel debugmdbctl
attack web_backend
dloglevel debugCLI Interface
Use debug logs to access the interface:
ipmcget/ipmcset --verbose=debugSNMP Interface
Currently, the SNMP interface does not support debug logs.
For the SNMP interface, the response body is organized by node, with relatively simple attribute meanings and straightforward usage scenarios. In most cases, faults can be located by analyzing the relationship between the SNMP interface and the resource collaboration interface.
Typical Log
Start/End of Interface Access
Start of interface access: Interface access begin, uri:%s, method:%s
End of interface access: Interface access end, uri:%s, method:%s
If the preceding logs are not found, the mapper configuration is not loaded. In this case, check the mapper configuration format.
The response result is 404:ResourceMissingAtURI, and the CLI interface outputs a command prompt.
Resource Collaboration Interface Error
Some errors are thrown by the service side: processing_flow_x, foreach_x, error_info:
Mapper Configuration Error
If some Statements fail to execute, the debug log will record the operation type and the error message. For example:
(Prefix-Add) The src data type is in correct, type = nilVerification and Interception
If some requests are intercepted in the verification phase and are not processed by the mapper, only one error message errs=error message is recorded, in addition to the logs indicating the start and end of interface access.
Fault Locating with the Resource Collaboration Interface Information on One-Click Info Collection
In the One-Click Info Collection page of openUBMC, each component directory under the AppDump directory contains an mdb_info.log file. If you can infer the corresponding resource collaboration interface through the mapper configuration or other locating methods, and know which component the interface belongs to, you can search for that interface in the corresponding mdb_info.log file.
This method can be used to locate the following faults:
- Component startup failure: The corresponding component directory is missing in the result of one-click information collection.
- Fault: Resource collaboration interface initiation failed. Symptom: In the one-click information collection result, the corresponding
mdb_info.logfile does not contain the expectedpath. - Fault: The attribute values of the resource collaboration interface are not as expected. Symptom: In the one-click information collection result, the interface attribute values displayed in the corresponding
mdb_info.logfile do not meet expectations.
In addition, you can use the resource collaboration interface attribute values obtained in this method to reversely infer the mapper configuration error.
Fault Locating by Modifying the Response Body
Although adding logs usually cannot help with fault locating in the mapper configuration, you can modify the configuration file to output its detailed processing flow to the response body, which aids in fault locating.
For example, the configuration of an interface is as follows:
{
"Resources": [
{
"Uri": "/redfish/v1/xxx",
"Interfaces": [
{
"Type": "GET",
"RspBody": {
"Name": "${Statements/GetName()}"
},
"Statements": {
"GetMetadata": {
"Input": "${ProcessingFlow[1]/Destination/Name}",
"Steps": [
"*" # * indicates an ellipsis. The same applies below.
]
}
},
"ProcessingFlow": [
{
"Type": "Method",
"Path": "*",
"Interface": "*",
"Name": "GetName",
"Params": [
"*"
],
"Destination": {
"Name": "Name"
}
}
]
}
]
}
]
}Modify the RspBody as follows:
{
"RspBody": {
"Name": "${Statements/GetName()}",
"TempName": "${ProcessingFlow[1]/Destination/Name}"
}
}After the modification, access the interface. The obtained response body is as follows. It can be inferred that the fault occurs during processing of the Statements:
{
"Name": null,
"TempName": "Name"
}NOTE
On the CLI interface, include TempName in the echo to display the temporary response for fault locating.
FAQs
Redfish/web_backend API Response Timed Out
After performing one-click information collection, check the 3rdDump/access_log file. If a timed out URI cannot be found, the fault is likely at the transport layer, such as environmental issues or inaccessible ports.
Error Type: 502
This error occurs in the following two situations:
- openUBMC is started but the northbound interface service is not started.
- The request format does not meet expectations. For example, a file is used as the request body in a non-upgrade interface.
Error Type: 412
This error usually occurs in the PATCH method of Redfish. Possible causes are as follows:
- After the
Etagis obtained, the resource attribute is modified. - The volatile attributes are not masked. As a result, accessing the resource twice may yield different values for the volatile attributes, causing the
Etagto change. - Response body disorder: When the object type attributes are retrieved directly from the resource collaboration interface, the key orders of the object may differ between two accesses, causing the
Etagto change. - Response body disorder: When the response body is spliced using a Lua table in the
Statements, the key orders of the table may differ between two accesses, causing theEtagto change.
CLI Interface Is Configured but Command Returns Prompt
Example: The /cli/v1/user/list interface is configured, but when the ipmcget -t user -d list command is executed, the command prompt of the -t parameter is returned.
Usage: ipmcget [-t target] -d dataitem [-v value]
-t <target>
... ...This problem typically occurs due to a missing configuration of /cli/v1/user. Add the configuration to solve it.