ESXI 6.5, Dell R510: HBA Passthrough, internal bay bypass

My recently acquired Dell R510’s purpose is ostensibly a backup storage array, along with backup services and a test bed for “production” (meaning my R710).

I’d previously been using Windows Server 2012R2 and Drivepool to great effect, so I figured a Virtualized version would work just as well. I’m mainly a Windows geek anyways, so ease of use wise, it’s great.

While I had replaced the Dell PERC H700 with an LSI 9211-8 HBA flashed to IT mode to allow direct passthrough, I still had to configure ESXI 6.5 to see it. The process to do this is not directly apparent, and I found out, not even possible using the HTML client ESXI 6.5 “exclusively” uses. I found out that while they say you only need the HTML client, you actually need the  Vsphere 6.0 2A Desktop Client to do most detailed operations. Eventually the desktop client will be transitioned out, but for now, definitely follow my link and get it loaded up.

Using ESXI  6.5 (without Vsphere Server) becomes quickly frustrating, since if you try to do regular management through the desktop client, it errors out (on 75% of the things) and points you to the html client. The html client however is incomplete, and you need the desktop client to do some things ( but not everything). The phased transition between the two has created a disjointed and frustrating experience. Moving on.

When I first started down the Server2012R2/Drivepool option, I configured the HBA (connected to the backpane) for direct passthrough, rebooted my host, and ended up losing all my stores and drive access.

See my mistake already?

All my drives, including my two internal SSDs holding my VM and ISO stores, are connected to the SAS2 backpane, which is connected to my LSI HBA, which I then configured for direct passthrough, to the VM that was on the SSD connected to the backpane… Circular problem here. Normally you’d say, Just use the internal SATA ports on the R510 motherboard, and move on. Easy and obvious solution right?

For some reason, Dell has disabled all the internal mobo SATA ports on the R510 12+2 configuration. I…I really cant figure out why. But its a thing apparently. On top of that, there are no internal power ports, Molex, Sata or otherwise. So no data, no power. How do you get two SSDs(or more) around the HBA connection then?

For data,  I installed a SATA expander card. I luckily had a two spare Sata Expander/controller cards I could use. Pop those into the PCIe slots, and I’ve solved half the equation. The other half was a little tricky, but absurdly simple after I figured it out, with the help of redditor (username) here(link).

I took pics of the process myself, hopefully they help you out:

Picture of the SATA expander that will host the SSDs separately from the HBA. There are a couple different holes you can thread the cables through to access the SSD caddies.

20170207_181418

The two blue cables are what you’re looking for. They’re the SAS ports for the SSD Caddy. Its also the only way to get any external power from the backpane.

20170207_181429

Now, there’s a couple ways to do this, the cable efficient way, and the ad hoc way:

1.) Order a SATA female to female Y split cable to plug directly into the SAS power port. You will need to clip a part of the female port that goes into the SAS, since its meant to fit over a SATA power port, not SAS.
2.) buy the wrong cable (Male to female), be lazy, and just connect it to the existing SAS cable as pictured.

20170207_184834

From there, you treat it like any other disk setup, and plug in the drives. here I’ve crammed 3 SSDs into 2 SSD slots. Not the best heat wise, but they have a dedicated fan blowing over them, Also, using this method, you could connect up to 2 more drives using 7 Splitters for power, and another expander/HBA. I’m not sure how much power you can draw through 1 SAS power port, but its possible you could daisy chain power off those two slots, and connect even more drives, IF you can find a place to stuff them. As a reminder, the internal SATA ports on 12+2 R510s are disabled. REALLY peeved about that.

20170207_184822

Space is very limited with the SSD bay, so if you can get right angle SATA and power cables, do it. Otherwise you’ll have to delicately cram them in and hope you dont pinch anything. Overhead is also limited, so make sure to stuff those cables down, otherwise you wont be able to close the lid.

20170207_185533

And, as always, after a hard days work on the servers reward yourself with a nice drink. My preference, a cool and tart whiskey sour made with fresh limes and lemons to cool off from the sauna in the server room (Next project, HVAC for the spare room…).

Happy Homelabing everyone,

-Garret

20170207_195608

 

 

 

 

 

 

 

 

 

 

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s