5 Node Cluster of Orange Pi Plus 2Es
Building a 5-node cluster for under £75 (£250 including five Orange Pi Plus2Es)After the interest in my cluster of Raspberry Pi 3s in April, I was keen to try building a cluster with some of the other excellent SBCs now on the market. Xunlong Software CO in China generously sent me 5 of their latest Orange Pi Plus2E ARM boards, each with an quad core ARM A7 SoC running at 1.3GHz, gigabit Ethernet and 16GB of eMMC flash storage onboard.
The completed cluster measures 146.4 (w) × 151 (h) × 216mm (d) and weighs 1.6kg (5.6 × 5.9 × 8.3", 55oz), including the AC-DC power supply.
Orange Pi Plus2E vs Raspberry Pi 3
The Orange Pi Plus2E board is a small but useful upgrade over the Raspberry Pi 3 both in terms of performance and features, and at an almost identical price:
| Model | Orange Pi Plus2E | Raspberry Pi 3 |
|---|---|---|
| SoC | 4-core ARM A7 1.3GHz2 | 4-core ARM A53 1.2GHz |
| Memory | 2GB DDR3 | 1GB DDR2 |
| Network | 1000Mbps | 100Mbps |
| WiFi | 802.11bgn | 802.11bgn |
| Bluetooth | no | 4.1 + BLE |
| Infrared | yes | no |
| Storage | 16GB eMMC onboard microSD card | microSD card |
| USB spare | 3 fitted 1 microUSB | 4 fitted |
| Video | HDMI HDCP/CEC, composite (CVBS) | HDMI, DSI |
| Camera ports | CSI | CSI |
| Audio | 3.5mm | 3.5mm |
| Microphone | onboard | no |
| Size | 94 × 60mm | 85 × 56mm |
| Power | 1.0 → 2.5W 3A max, DC 4/1.7mm | 1.2 → 2.1W 2.5A max, microUSB |
| Launched | Q2 2016 | Q1 2016 |
| Price (UK) | £33.251 | £33.59 |
1US$35 Plus2E + $4.16 shipping + 20% VAT + 0% import duty = £33.25 | ||
3D design in SketchUp
I modified my original Raspberry Pi cluster design using the free version of SketchUp and built rough 3D templates of the Orange Pi Plus2Es, network switch, sockets, etc. I didn't bother to include ventilation slots/grids in the 3D model.
2D design in Inkscape
I used the free Inkscape application for 2D design, ready for exporting to the laser cutter. Each colour is a different pass of the laser, at different power/speed levels, so the green lines are cut first to make holes for ports/screws/ventilation, pink are extra cuts to help extract delicate parts, orange is text/lines that are etched and finally blue cuts the outside of each panel.
Download files for laser cutting on one 600×400×3mm sheet:
Design Changes from the Raspberry Pi 3 cluster
Although very similar to my cluster of Raspberry Pi 3s, I made a number of changes and improvements:- I kept the horizontal mounting rail design but the Plus2E has M3 holes which are easier to find parts for, rather than M2.5 on the Pi. And the holes are further apart because the overall board size is bigger than the Pi. Screwing the plastic nuts onto horizontal rails is a bit tedious (and a tight squeeze to get past the ethernet port on each Plus2), and I'd like to 3D print a C-shaped ‘clip’ that holds the boards in place along each rail, or perhaps use tight elastic washers.
- External PSU vs Internal USB hub – the Plus2Es are powered via 4mm/1.7mm DC plugs rather than the Pi’s microUSB, so it made sense to swap out the internal USB hub power supply for an AC-DC power ‘brick’ that sits outside the case. This gives more space inside the case (for the larger Plus2E boards, and a fan), and should help with dissipating the heat from the power supply. Each Plus2E can draw up to a maximum of 3A, but will actually be much less in this cluster, without extra USB and GPIO accessories.
- DC cable splitter. I used a cheap 45cm 6-way splitter cable sold for CCTV cameras is around £2, and cutting off the (wrong size) DC plugs and soldering new right-angled 4mm/1.7mm DC plugs gives a neat result. The 6-way cable also allows for a 5V connection to power the ethernet switch, saving on the cost of a separate cable.
- Internal Case Fan vs External – I wanted to experiment with different clock speeds for the Plus2E boards which would definitely need active cooling, and so I stuck with an ultraquiet 92mm fan, but moved it inside the case, and to the back rather than on top.
- Case USB ports – while perfectly functional, I never really liked the look of the twin USB ports on my Pi cluster, so for the Plus2E cluster I sourced a more compact 2-port USB mount.
- No internal shelf vs Shelf – not needing a shelf to attach the USB hub to simplifies the design, and also means it can be cut from a single sheet of 600×400mm acrylic. Removing the shelf would have reduced the rigidity of the case, but screwing the horizontal mounting rails to the side panels keep it secure.
- Orange LAN cables vs Boring grey – could they really be ANY other colour??
I initially bought 25cm cables, but it was tight bending these inside the case, so I swapped them for 12cm cables. I prefer the ‘flat’ network cables that are easier to bend, but I couldn't source any short cables in orange. - Gelid Solutions Silent 9 fan vs Nanoxia Deep Silence fan – I was very happy with the performance of the Nanoxia fan (and their excellent customer service) but wanted to try out a cheaper option. The Gelid rubber grommets are thicker than the Nanoxia ones, so I increased the case mounting hole diameters by 0.5mm.
- Left facing boards vs Right facing – this just meant access to the HDMI and SD card ports were facing upwards on the Plus2Es, and easier to access.
- Gigabit vs 10Gigabit-switch – the Plus2E network ports are each 1000Mbps (10× faster than the Pi), so using at least a 1000Mbps switch is a no-brainer really. A 10Gbps switch would eliminate any bottleneck (e.g., if the Plus2's were simultaneously saturating their links talking to the outside network), however these are still very expensive at £600+, and too large to fit inside this case. The new Asus XG-U2008 switch looks promising.
Power, temperature & cooling
At idle, the entire system of five Plus2Es, network switch & 7V fan sips a mere 9W, and at 100% load (1.3GHz) it still only uses 18W in total. There is the possibility of further reducing the power requirements by disabling GPU/HDMI, USB and WiFi?1.3GHz or 1.6GHz? The maximum speed of the Orange Pi Plus2E is advertised as being “1.6GHz” but Armbian Linux limits this to 1.3GHz. The fastest I managed to run the boards was 1.536GHz, which requires driving the SoCs at 1.5V instead of 1.32V – this uses 60% more power (4.0W vs. 2.5W per board) and needs more cooling to get rid of that added heat. Even with a 12V fan running full speed (which is noisy), heatsinks on each board and plenty of ventilation, my cluster of 5 boards at 1.536GHz would reach throttling temperatures after only 1 minute, so I gave up on the overclocking.
Is a heatsink worth it? the Plus2E SoC does generate a bit more heat than the Pi3, so a heatsink is probably a good idea. Xunlong sell a strange “heatsink” product that appears to be nothing more than a thin piece of 3M heat conduction tape? Instead I used third party aluminium heatsinks that come with (3M!) adhesive tape. These do reduce the temperature by a few degrees, but they aren’t nearly as effective as the jumbo clip-on heatsinks that the NanoPC-T3 board uses.
The power adaptor can supply up to 50W (2A per Plus2E), which is more than enough for a couple of power-hungry external devices to be plugged into the USB ports. Using:
armbianmonitor -mto measure the SoC core temperature, the cluster idles at 25°C (77°F) with cooling from the 7V fan.
At 100% load and the 7V fan, using:
sysbench --test=cpu --cpu-max-prime=200000 --num-threads=4 run &the SoC core temperatures reached a stable 54°C (129°F). If you don't provide at least some active cooling, the SoCs will reach 70°C within a minute and then automatically throttle down their clock speed, to avoid overheating. They can safely run long-term at that temperature, but you don’t get maximum performance.
Silent cooling:
To cool down the cluster I fitted a single 92mm fan inside the rear of the case. I used a (effectively) silent fan recommended by Quietpc.com, the Gelid Silent 9 (£5.53).At 5V I have to get my ear within 50-75mm (2-3") to hear even the slightest whisper from the fan, and the supplied rubber gromets definitely do a good job of isolating the case from any small vibrations. However at 12V the fan is quite audible in an otherwise silent room, so I was looking for the voltage that would provide enough cooling, while keeping the fan silent.
For a single Plus2E you might not need a fan at all, or perhaps a small 40-60mm one.
| description | heatsinks? | idle | 100% load | performance |
|---|---|---|---|---|
| Case, no fan, 1.3GHz | no | 38°C | 74°C | throttles |
| Case, 5V fan, 1.3GHz | no | 31°C | 68°C | throttles (just) |
| Case, 7V fan, 1.3GHz | yes | 25°C | 54°C | OK |
| Case, 12V fan, 1.534GHz | yes | 17°C | 70°C | throttles (just) |
Unlike PC motherboards, a Plus2E doesn't have a standard 3 or 4-pin fan header, but it is quite easy to power the fan using the GPIO 5V power rail instead. Note that unlike on the Raspberry Pi, the power rail is wired directly to the board DC input, so there should be no problem using it to drive even power-hungry add-ons.
Most fans won't work (or will spin very slowly) at 5V, but you can use a small boost-convertor to ramp up that voltage to anywhere between 5V and 12V. This takes the 5V output from the Plus2E, and boost it to between 5V and 12V to set a different fan speed. There is a tiny screw on the board to adjust the output voltage, but use a multimeter to check the output voltage before connecting to the fan, as this particular converter can output up to 28V which would burn out the fan 
. It only costs £2.04.
Building the Plus2E Cluster
- Remove network switch case (2 small screws)
Attach the network switch to case base, using 4× 6mm brass spacers + 4× 14mm bolts + nuts. This only fits one way around - Cut off only 5 (of the 6) plugs on the DC power splitter cable, and solder on 5 new 4.0/1.7mm right angled plugs
Bolt DC socket port to case back
Cut off the combined end of the splitter cable, and solder onto the DC screw mount socket.![Building Plus2E Cluster case 2]()
- Screw 2 external LAN ports to inside of the case back (has “DC5V 10A” etched on the outside)
Fit the 92mm case fan to inside of the case back using the supplied rubber grommets. The fan logo should face inwards if you want the fan to blow cold air into the case. The Gelid rubber grommets are a VERY tight fit and might need to be pushed into place with a small screwdriver/pen as well as pulling quite hard!
Clip case back to case base (marked C+D)
Plug external LAN into network switch ports 1+3 (no room to use 1+2)![Building Plus2E Cluster case 3]()
- Stick a heatsink onto the SoC chip of each Plus2E
Slide 5 Plus2Es onto 4 threaded rods using 48 nuts to secure. This might be faster with the rod in an electric screwdriver? Leave 30mm space at left end, space each Plus2E 25mm apart. The LAN+USB side of the Plus2Es point towards the front of the case, with the HDMI and SD card slot pointing upwards
Attach case sides to Plus2E rods (EF+AB), secure with 8 metal nuts.![Building Plus2E Cluster case 4]()
- Cut the fan cable in half, and solder the red & black wires to the boost-convertor circuit. (The VIN pins go to a Plus2E, and the VOUT pins continue to the fan.) The yellow wire isn’t used.
Fit the 3-pin fan plug into the GPIO pins of one of the Plus2Es, so that black (ground) connects to pin 6, red (power 5-12V) to pin 2, and yellow (fan speed) is unconnected. To remove the individual wires from the plug, depress the tiny metal tabs (e.g., with a small screwdriver), and the wires will slide out. I cut off the yellow wire which I wasn't interested in, and slid the wires back into the plug so that red was on the left, the middle was empty, and black was on the right:
![Building Plus2E Cluster case 5]()
- Plug Plus2E LAN cables into network switch ports 4-8 (be very careful if removing these – it is easy to break tiny plastic clips on the switch ports)
Clip case sides to case bottom (E+F, A+B)![Building Plus2E Cluster case 6]()
- Route HDMI cable through case
Screw external twin USB to front case
Screw external HDMI to front case
Plug 2 external USBs into any Plus2E (or 2 separate Plus2Es)
Plug LAN cables into Plus2Es
Plug DC power cables into Plus2Es
Plug HDMI cable + right-angle adaptor into any Plus2E![Building Plus2E Cluster case 7]()
-
Clip case front to case bottom (G+H)
Clip case lid to case sides, front & back (M+N+O+P+Q+R+S+T)
Attach self-adhesive rubber feet to underside of base
Bill of materials
Most of these parts were sourced from individual sellers on Amazon or eBay, which of course racks up the postage charges. If there were enough demand, it would be cheaper to bulk buy the parts and have a kit with everything you need to build the cluster.
| Edimax ES-5800G V3 Gigabit Ethernet Switch | £13.49 |
| 12cm Cat5e LAN cables (5 pack) | £4.75 |
| 5V 10A AC-DC external power supply, 5.5/2.5mm plug | £12.99 |
| 5.5/2.5mm chassis mount DC socket (1 from a 2 pack) | £1.95 |
| 6-way DC power splitter cable | £2.05 |
| DC plug right angled 4/1.7mm (5 pack) | £3.49 |
| M3 steel screws 8mm (4 from a 10 pack) | £1.10 |
| RJ45 male to female screw mount (2 pack) | £1.98 |
| M3 steel hex nuts (4 from a 5 pack) | £1.10 |
| M3 steel bolts 14mm (4 from a 5 pack) | £1.35 |
| M3 brass female standoff 6mm (4 from a 10 pack) | £0.99 |
| M3 steel threaded bar 150mm inc. nuts (4 from a 5 pack) | £5.50 |
| 0.5m HDMI male to female panel mount (inc. bolts) | £2.39 |
| Twin USB female socket to male cable (inc. screws) | £2.19 |
| M3 nylon hex nuts (48 from a 100 pack) | £1.29 |
| 3mm extruded clear perspex 600×400mm | £5.32 |
| Laser cutting charge | n/a |
| HDMI 270 degree adaptor | £0.99 |
| Gelid Silent 9 92mm case fan | £5.53 |
| Mini 5V-to-12V step-up/boost converter | £2.04 |
| Polyurethane rubber feet (4 from a 12 pack) | £2.99 |
| Subtotal inc P&P | £73.48 |
| Orange Pi Plus2E at US$35/each (5 pack)1 | £161.65 |
| Raspberry Pi aftermarket heat sinks 5mm (5 pack) | £1.09 |
| Total inc P&P | £236.22 |
1The Orange Pi Plus2E is duty free to import into the UK, costs US$15.43 to ship 5 boards from China, and there is UK VAT at 20% bringing the total to £161.65.
The cluster of Plus2Es is overall slightly cheaper to build than the Pi3s: you don't need a USB hub, separate microUSB cables, SD cards for each board, etc. However it is a shame they used 4/1.7mm DC power connectors, rather than the far more common 5.5/2.5mm or 5.5/2.1mm sizes.
Show me the Benchmarks!!
I'm writing a separate article benchmarking this Orange Pi Plus2E cluster against my original Raspberry Pi 3 cluster, as well as detailed benchmarks comparing the individual boards. Watch this space...Clusters of other Single Board Computers
So far I’ve built clusters using the following ARM boards:- DIY 5 Node Cluster of Raspberry Pi 3s
- 40-core ARM cluster using the NanoPC-T3
- 5 Node Cluster of Orange Pi Plus 2e
- Bargain 5 Node Cluster of PINE A64+
- ClusterHAT with 4× Raspberry Pi Zero
- 96-core ARM supercomputer using the NanoPi-Fire3
I’d like to build a small cluster of all the current crop of sub-$100 ARM SBCs, comparing the different features, and with detailed benchmarks. e.g., Odroid C2/XU4 and the Banana Pi M3. Please email me if you'd like to send boards for review.
Software to run on a cluster?
or... What is it for??Education, training, blah, blah... well personally I’m just running Armbian (Debian Linux) on each Plus2E for now, and I’m going to experiment with things like load-balanced web/database servers.
Running Docker on ARM on each node looks like an excellent way of controlling the cluster.
Nick Smith, July 2016.
