So how is Dark Melee on a Tanker?

Whilst Tanker Dark Melee is probably the highest ST damage-over-time set and is certainly not anemic, it's not as good as it is on Scrappers or Brutes.

As you've touched on in your post, this is due to the Tanker lower damage modifier (compared to Scrappers) and the Tanker lower damage Cap and lack of Fury (compared to Brutes - Tankers actually have a higher base modifier than Brutes) - You're simply never going to be able to equal the prolonged damage output of those ATs, with sets and slotting being equal.

What Tanker DM can do however is come pretty close to matching the prolonged damage output of other Brute/Scrapper Primaries, at least in terms of Single Target damage output. And particularly with the new Bruising -20% resistance debuff on Tanker Tier 1 attacks - as Shadow Punch is actually pretty decent DPA and fits nicely into one of Dark Melee's attack chains.

The chain I've personally settled on is Gloom->MG->Smite->SL->SP->Smite - not the most optimal one on a Scrapper or Brute, but on a Tanker it works nicely at just over 9.1 seconds for a full cycle meaning SP's Bruising effect is always active. This inflicts an average of a little over 58 raw DPS unslotted . All things considered, full IO build and everything (and SL slotted for Healing as well as DPS)... I work out my Elec/DM Tanker's maximum damage output to be a little over 217 DPS with a Saturated Soul Drain and his Damage Aura running, or a little over 153 DPS with only one mob within Soul Drain range. For comparison, one of my Mace/Willpower Brute builds with a similar level of IO investment running an optimised attack chain hits an average of 47.94 DPS unslotted, a little over 184 DPS slotted and at 75% fury, and a little over 222 DPS with 75% Fury and Build Up running. Now whilst this might be nothing compared to the 290+ DPS that certain high-end DM/Shield Scrapper builds can put out, it's not exactly "poor", considering that it takes just over 94 DPS to solo an Archvillain (or at least to exceed their natural regeneration).

In the above example, the Tanker is actually better DPS over time than the Brute, assuming that you can keep enough mobs nearby to fully leverage 'Soul Drain'. The reason for this is that Tankers gain much more from similar levels of damage buffs than Brutes due to their higher base modifiers - and saturated Soul Drain really is that powerful... and as an added bonus, the -20% resistance from Brusing is making your entire team hit harder as well. The trade-off for this is poor AoE damage, and the need for mobs to fuel Soul Drain.

In all honestly, DM tends to go well with pretty much anything (aside from possibly Willpower, since DM likes more recharge to get Soul Drain up more often and Willpower gains nothing from +Recharge).

Dark Melee's big negative on teams is the lack of AoE damage output - for a Tanker, this reduces your aggro control as well as making you more effective versus bosses than hordes of minions. For that reason DM tends to work better on teams if you can grab AoE attacks from elsewhere - either your Primary or the Epic pools.

Of the three primaries you mentioned, I'd go with FA for extra spike Damage and the AoE potential of the Damage Aura + "Burn", Shield for extra prolonged Damage via AAO (and some spike AoE damage in Shield Charge) and INV for maximum survivability. DM's heal will help a lot if you go INV/, and Soul Drain works well with Burn/AAO/Shield Charge, so they all have some good synergy going.

You might have issues with survivability tanking unaided for teams as a Fire/DM. My own Fire/Energy (with "Tough") tended to faceplant a bit as a main Tanker for full teams until I slotted him up for S/L Defence set bonuses. A softcapped Shield/DM won't encounter much that can put it down in standard PVE except for Devouring Earth, Nemesis and possibly Romans. A well-build INV will laugh at pretty much anything except for certain Psionic AVs or hordes of Carnies/Malta.

Note also that Dark Armor is also a very good pick here as a "middle ground" option - with a bit of investment into Defence it can get nearly as tough as INV (and tougher versus Psi foes) but it comes with a Damage Aura and Endurance Drain resistance. It's downside is that it gobbles Endurance, but DM comes with a decent endurance recovery tool.

I personally went with Elec/, which is a bit of a Dark Horse set in that whilst it has no real holes in its typed damage mitigation it's usually not as tough as Dark or INV... but it does have a lot of other "toys" to play with - Regen/Heal ability, an Endurance Recovery tool, a Damage aura, a +Recharge passive, End Drain Protection and the ability to Drain foes of their Endurance. For me, DM provided extra healing potential and a bit of -tohit to layer on top of all my damage resistances (I managed to hit over 90% S/L/E, 50% N, 65% F/C and 70% Psi).

I am having a lot of fun with a Fiery Aura/Dark Melee Tanker. It's clicky; unlike a lot of Tanker builds you can't just let the tanker stand there while you mash your attacks. You've got 2 heals which are critical to keeping yuo standing; potentially 2 endurance stealing powers (not everyone takes both, though you could), and Burn and Fiery Embrace to consider when to click as well. Then there's positioning Shadow Maul's cone. But it feels like a "good busy" instead of a "bad busy" situation -- properly leveraging these tools makes for a pretty unusual but effective Tanker.

Psi trouble I understand...but why would Inv have problems with the two factions? I can't recall having had any issue with them myself. Remember that if you take one of the passives, Inv has decent end drain protection.

For starters, Resist Energies only provides +25% end drain/recovery protection. This isn't a huge amount (Dark Armor, for comparison, gets over +85% protection) and functions more as a means of prolonging the time it takes to become drained than a means of actually preventing it from happening.

Versus Carnies, aside from the Psi damage and hold spamming (notably versus Master Illusionists) INV is toggle heavy and the Ring Mistress "Mask of Vitiation" power basically negates endurance recovery for its duration. It's a huge debuff, and Resist Energies certainly won't prevent it from completely stopping your recovery. If you're soloing and don't encounter a lot of Bosses, Carnies are manageable (Illusionists can be annoying with all the phasing but they're no real threat, and the -endurance they all do on dying is more annoying than dangerous).

Versus Malta, Sappers have a spammable -Endurance power which is quite capable of draining /Dark dry with repeated applications, let alone an /INV. Malta Sappers are only meant to appear once per regular mob and tend to be OK if you target them early and take them out via damage or mez, but "ambush" or "rescue" mobs can add to this considerably.

I also didn't mention Rularuu. Virtually everyone has trouble with Rularuu. INV at least has its resists to fall back on when confronted with Eyeballs, but all the stacked debuffs from an aggro cap's worth of Brutes, Natterlings and Eyeballs combined with Wisps Psi damage output can overwhelm even a careful INV tanker.

My own main endgame Tanker is an IO'ed INV/SS, and whilst I don't predict a well-built INV will have overwhelming problems if they're soloing or tanking these type of mobs for small teams, when facing an 8-man-team's worth of higher-level foes they will likely start being ground down (at least without support), even with Softcapped typed defences. And even when soloing, these foe types will likely make an INV start to pay attention to what they're doing a bit more than usual.