pciefw_Loader_install.txt (12417B)
1 PCIe Firmware Loader Introduction 2 =================================== 3 4 Broadcom CMICx based devices have a PCIe SerDes embedded. A special 5 firmware provided by the Broadcom SerDes team should be loaded into the 6 SerDes core to configure and bring up the PCIe SeDes link up to Gen3. A PCIe 7 firmware loader is designed for running on a CMICx embedded CPU, M0 core 2, to 8 load the PCIe firmware and bring-up the PCIe SerDes with taking care of PCIe 9 standard timing requirements. 10 11 Sometimes, PCIe Gen2 or Gen1 also requests PCIe firmware to handle some 12 SerDes hardware WARs and/or get better timing for performance. 13 14 The loader is also used for some switch hardware WARs, which should be taken 15 care of at a very earlier stage of system boot up, but those could be not 16 PCIe related. 17 18 In applications, there could have some interoperation issues, such as special 19 clock configurations required, on a customer's board design. 20 21 PCIe firmware loader provides an approach for customers to define some 22 customizing settings attached at the end of the loader image and PCIe loader 23 firmware will take care of those configurations. There are three classes of 24 customizing settings: controls, pre-configs, and post-configs. The controls 25 are used for control PCIe firmware loader running flow (such as ignore PERST 26 timing for the first loading) or common settings (such as the number of lanes 27 and PCIe generation). Normally, controls are used for debugging purposes or 28 temporary workarounds. The pre-configs and post-configs are used for 29 configuring PCIe MAC and/or PHY registers before (pre-configs) loader boot-up 30 PCIe SerDes or shortly after (post-configs) the boot-up. Mostly, the 31 configurations are the customer board special requested. (Refer to "How to 32 apply custom/user settings" section for format and usage) 33 34 There are four PCIe firmware loader images currently supported for different 35 switch classes. (vXpY is version number, e.g. v2p10 means version 2.10) 36 a) pciefw-m0_vXpY.bin: Include 16-nm SerDes firmware, boot from M0 CPU core 0 37 (for switches without R5 CPU(s) embedded). E.g. Helix5. 38 b) pciefw-r5_vXpY.bin: Include 16-nm SerDes firmware, boot from R5 CPU core 0. 39 E.g. Tomahawk3. 40 c) pciefw-r5-m7_vXpY.bin: Include 7-nm SerDes firmware, boot from R5 CPU core 0. 41 E.g. Tomahawk4. 42 d) pciefw-d5-m0_vXpY.bin: Include 5-nm SerDes firmware, boot from M0 CPU core 3. 43 E.g. Tomahawk5. 44 45 The following sections will elaborate on how to install a PCIe firmware loader, 46 check running status, and add custom settings for CMICx based XGS devices. 47 48 Install PCIe Firmware Loader 49 =================================== 50 There are a couple of ways to program a PCIe firmware loader into a serial 51 flash for PCIe link bring-up. The user may choose one of below approaches: 52 53 Approach 1: Using flash/gang programmer 54 The user may use a standalone flash programmer or a gang programmer to burn 55 a PCIe firmware loader image into a serial flash by following the programmer's 56 instruction. To program the flash, it should be inserted on the flash bed of 57 the programmer. So, the programming should be done before soldering it onto a 58 hardware board or uninstall it from the board. 59 60 Approach 2: Using ARM DS-5 ARM DSTREAM 61 A small tool was designed to run on an internal R5 CPU core in Broadcom switch 62 for programming a PCIe firmware loader into the serial flash on the board. The 63 user needs to connect an ARM DSTREAM onto the board through a JTAG interface, 64 run ARM DS-5, load the tool with the loader image onto R5 workspace, and run 65 the tool on the R5 through DS-5. The tool will program the loader image into 66 the flash device. This method works only for the switch which has R5 CPU 67 embedded. (Refer to tools/flashwrite/ARM_DS-5 for details.) 68 69 Approach 3: Using SDK diag command 70 This approach uses SDK "PCIEPHY" diag command to load a PCIe firmware loader 71 image into the flash on the board. It requests PCIe link has already 72 established stably and SDK may access the flash. In case PCIe link could not 73 be established stably, that may be triggered by an abnormal PCIe firmware 74 loader (or blank flash), the customer may use board hardware switches to 75 configure PCIe generation force to Gen2 (or Gen1), which may get link up 76 without PCIe firmware loader supported. (About how to force PCIe Gen2 77 (or Gen 1), please refer to the corresponding device datasheet. ) After PCIe 78 firmware loaded, restore the force setting to enable PCIe firmware to select 79 the best link speed, and reboot the board run the loader. 80 81 SDK6 PCIEPHY command format: 82 pciephy fw load <Firmware file> 83 e.g. "pciephy fw load pciefw-r5_vXpY.bin" 84 85 SDKLT PCIEPHY command format: 86 pciephy fwload <Firmware file> 87 e.g. "pciephy fwload pciefw-r5-m7_vXpY.bin" 88 89 Notes: Refer to the "Instruction" section and make sure to select a correct 90 loader image for the device. 91 92 The flash driver in SDK is based on MT25QL256. If the mounted flash part is 93 compatible with MT25QL256's instruction set but unrecognized by the driver, 94 the user may use SoC property "pcie_flash_mem_params" to specify the 95 parameters of the flash device in SDK6. 96 pcie_flash_mem_params=<id>,<sector_size>,<page_size> 97 <id>: Manufacturer ID. 98 <sector_size>: The smallest erase unit in bytes. 99 <page_size>: The smallest access unit in bytes. 100 For SDKLT, please use below command instead: 101 spi flashdev id=<id> size=<total_size> pagesize=<page_size> 102 sectorsize=<sector_size> 103 e.g. 104 "spi flashdev id=0x9d size=0x8000000 pagesize=256 sectorsize=0x10000" 105 106 Boot and verify PCIe Firmware Loader 107 =================================== 108 1. Ensure that strap signals are set to load the firmware: 109 a. Mhost should boot from QSPI 110 (e.g. BOOT_DEV[2:0] (if available) is pulled {low, low, low}) 111 b. Mhost should be brought out of reset 112 (e.g. MHOST0_BOOT_DEV (if available) is pulled high) 113 c. PCIE force generation should be clear 114 (e.g. PCIE_FORCE_GENTYPE[1:0] (if available) is pulled {low, low}) 115 The signals/registers names could vary among switch devices. Please refer 116 to the switch's datasheet for the appropriate signal names. 117 118 2. Power On the system 119 120 3. Load and run linux 121 a) At a linux prompt run 122 lspci -d 0x14e4: -vvv 123 124 b) The Broadcom switch device should have enumerated and should show 125 something similar as the following: 126 "05:00.0 Ethernet controller: Broadcom Limited Device b870 (rev 01)" 127 128 c) In the lspci output, check that the link status register field shows 129 link-up speed matches your expection. 130 32GT/s - PCIe Gen5 131 16GT/s - PCIe Gen4 132 8GT/s - PCIe Gen3 133 5GT/s - PCIe Gen2 134 2.5GT/s - PCIe Gen1 135 136 d) check that link capability register shows "ASPM not supported" and 137 link control register shows "ASPM disabled" 138 139 e) Run SDK and verify that "pciephy fw info" ("pciephy fwinfo" in SDKLT) 140 shows firmware loaded and both loader version and firmware version 141 match your expectation. 142 e.g. PCIe FW loader version: 2.8 143 PCIe FW version: D102_0B 144 PCIe FW loader built date: 20200706 145 Firmware was loaded! (version: 0xd20b) 146 147 f) Run "tr 502" in SDK and get pass. 148 149 Apply custom/user settings 150 =================================== 151 It is anticipated that some of MAC/PHY settings, like TX FIR etc, may be tuned 152 by customer based on customer system requirements. PCIe Gen3 firmware loader 153 offers provision for such cases. This provision offers optional custom settings 154 programming before or after loading SerDes firmware. This provision requires 155 custom or system specific PCIe MAC/PHY register settings to be programmed at 156 30K offset of QSPI flash in the format described below i.e 157 {type, address, value} triplets. 158 159 ----------------------- __ 160 | 0x50434945 | | 161 | 0x41524753 | |__ Header, fixed, 4x32-bit words 162 | 0x00000000 | | 163 | 0x00000000 | __| 164 ----------------------- __ 165 | 0x4354524C | | Optional controls for FW load 166 | controls_0 | |__ Must have 0x4354524C as first word 167 | controls_1 | __| followed be two controls words 168 ----------------------- __ 169 | 0x5052454c | | 170 | {type, addr, val} | | 171 | {type, addr, val} | | Optional pre FW load settings 172 | : | | Must have 0x5052454c as first word 173 | : | |__ followed by {type, addr, val} 174 | : | | triplets (triplet - 3x32bit words) 175 | : | | type: 0 - MAC address, 1 - PHY addr 176 | : | | addr: MAC/PHY register address 177 | {type, addr, val} | __| Val: Value to be programmed 178 ----------------------- __ 179 | 0x504f5354 | | 180 | {type, addr, val} | | Optional post FW load settings 181 | : | | Must have 0x504f5354 as first word 182 | : | |__ followed by {type, addr, val} 183 | : | | triplets (triplet - 3x32bit words). 184 | : | | type: 0 - MAC address, 1 - PHY addr 185 | : | | addr: MAC/PHY register address 186 | {type, addr, val} | __| Val: Value to be programmed 187 ----------------------- __ 188 | 0x00454E44 | |__Footer, fixed, 2x32-bit words 189 | 0x41524753 | __| 190 ----------------------- 191 192 User can program up to maximum 168 triplets (i.e 504x32-bit words). 193 There is no restriction on number of triplets for pre FW load settings or post 194 FW load settings, provided total number of triplets do not exceed max limit 195 of 168. 196 197 Control section is used for optionally disable some loader feature or change 198 basic settings. Currently support below controls: 199 control_0[0:0] - Ignore enabling hotswap 200 control_0[1:1] - Ignore PERST status initial check 201 control_0[7:4] - Link Speed (PCIE generation) override, 202 1 - 2.5G, 2 - 5G, 3 - 8G, 4 - 16G, 5 - 32G, 0 - unconfigured 203 control_0[11:8] - Link width override, 1, 2 or 4 lanes are valid 204 Other fields are reserved for future. 205 206 Each type customer settings are optional, and total support up to 168 settings. 207 The settings should be organized as the below sequence. 208 [Controls] -> [Pre] -> [Post] 209 210 The address[26:16] of lane based PHY registers is used for lane number 211 assignment. 0 ~ 3 are used for selecting lane 1 ~ 4, and 0x1ff means 212 broadcast configuration to all PCIE lanes. 213 214 A sample format file in text format (custom.txt) and perl based text to binary 215 conversion utility provided under tools/flashwrite/custom/ directory of this 216 release package to help customer to generate customer settings in binary format. 217 218 Customer settings in text format may have comments with '#' at the begining of 219 the line and also blank lines which will be ignored by text to binary conversion 220 tool. 221 222 Syntax of text to binary conversion tool: 223 perl custom_ascii2bin.pl <input text file> <output file> 224 225 Example: Conversion of sample custom.txt file 226 227 linux#: perl custom_ascii2bin.pl custom.txt custom.bin 228 454943505347524100000000000000004C52544330040000000000004c45525000000000381800000040088254534f5001000000b0d001008024000001000000b1d0ff0107000000444E450053475241 229 230 Hexdump of generated file should look like the following: 231 linux#: hexdump -C custom.bin 232 00000000 45 49 43 50 53 47 52 41 00 00 00 00 00 00 00 00 |EICPSGRA........| 233 00000010 4c 52 54 43 30 04 00 00 00 00 00 00 4c 45 52 50 |LRTC0.......LERP| 234 00000020 00 00 00 00 38 18 00 00 00 40 08 82 54 53 4f 50 |....8....@..TSOP| 235 00000030 01 00 00 00 b0 d0 01 00 80 24 00 00 01 00 00 00 |....°Ð...$......| 236 00000040 b1 d0 ff 01 07 00 00 00 44 4e 45 00 53 47 52 41 |±Ðÿ.....DNE.SGRA| 237 00000050 238 Notes: The "hexdump" command shows data in 4-byte little endian format, while the data 239 is 4-byte big endian, e.g. "0x50434945" shows "45 49 43 50". 240 241 Installing custom settings binary file: 242 1) Append binary file to PCIe Gen3 FW file (pciefw-*.bin) 243 e.g., 244 cat pciefw-r5_vXpY.bin custom.bin >custom_pciefw-r5_vXpY.bin 245 2) Install the appended binary into QSPI flash at offset 0x0 using one of the 246 FW installation methods mentioned above.