wip: replay of trampoline forwards

[carlaKC]

This PR contains the most data-heavy / least-smart approach to handling replay of trampoline forwards. Opening this up as a basis for discussion, not intended for real review!

There's a lot of background written up here - for TLDR skip to ➡️section⬅️ which has the open question I'm unsure about.

---

Trampoline Background

Trampoline forwards may have multiple HTLCS incoming and outgoing, for example:

A --
        -- D
B -- us
        -- E
C --

On our outbound channels (D/E), we represent these forwards using a HTLCSource which contains information about all of our inbound HTLCs and the part of the payment dispatched on that outgoing channel.

In D's pending_outbound_htlcs:

HTLCSource::TrampolineForward
  previous_hop_data: [A, B, C]
  outbound_payment: { payment_id: X, session_priv: P1, path: {D, ...} }

In E's pending_outbound_htlcs:

HTLCSource::TrampolineForward
  previous_hop_data: [A, B, C]
  outbound_payment: { payment_id: X, session_priv: P2, path: {E, ...} }

Note: each outbound HTLC tracks all inbound, and they have a shared payment_id but different session_priv!

Claims:

• We call claim_funds_from_htlc_forward_hop for each prev_hop reported by HTLCSource.
• We use our previous_hop_data to create our PaymentForwarded event.

Fails:

• We use our trampoline payment_id in fail_htlc_backwards_internal to check whether it's appropriate to fail back at this time.
• We push a forwarding failure for each prev_hop reported by HTLCSource.

---

Replays

We populate already_forwarded_htlcs from all of our inbound htlcs (inbound_forwarded_htlcs), and de-duplicate any HTLCs that still present on outbound channels (outbound_htlc_forwards). Anything that is remaining has a claim replayed when a preimage is present on the inbound monitor, and a fail otherwise. We track our inbound htlcs per-inbound channel, and keep a record of the outbound htlc that they are associated with in InboundUpdateAdd.

In our multi-in, multi-out trampoline world there are a few things that we need to consider:

➡️ (1) Reconstructing multi-inbound ⬅️

Right now, we reconstruct the HTLCSource::PreviousHopData from InboundUpdateAdd and the channel monitor that our inbound htlc is on. We can't do this for trampoline because our source is spread across multiple inbound channels.

Our options here are:
a) Store information about all inbound HTLCs in each InboundUpdateAdd so that we can reconstruct our HTLCSource
b) Store information about only our channel's inbound HTLC in InboundUpdateAdd, and post-process our already_forwarded_htlcs to collect HTLCSource by payment_id
c) Store information about only our channel's inbound HTLC in InboundUpdateAdd and do not reconstruct multi-in `HTLCSource (this seems like a bad idea, but including it for completeness).

This PR implements (a), which has the downside of being very data-heavy. The downside of (b) is that we could produce events with missing information when some of our inbound HTLCs have been cleared out, but it doesn't seem terrible.

❓ I think that it's worth taking a look at the complexity of (b) but would be interested in thoughts here!

(2) Multiple OutboundHop(s)

As we commit each of our outbound HTLCs, we'll report the HTLCSources above for D and E in committed_outbound_htlc_sources and use them to prune_persisted_inbound_htlc_onions. We track each unique session_priv's OutboundHop so that we're able to get a full record of all the outbound HTLCs that our inbound is associated with. If any of our outbound HTLCs are still present, we'll appropriately prune the inbound from already_forwarded_htlcs.

I don't think there are many different approaches to this - we could have a 1-in/2-out trampoline forward, so we need to track all outbound with our inbound. This means we'll have some duplication when we have multiple inbound HTLCs.

[ldk-reviews-bot]

👋 Hi! I see this is a draft PR.
I'll wait to assign reviewers until you mark it as ready for review.
Just convert it out of draft status when you're ready for review!