Using FnVoice with a different networking stack (PUN2 / Fusion 2 / any transport)

FnVoice's audio pipeline (mic capture -> VAD/APM -> Opus encode -> jitter/PLC decode -> playback) is decoupled from FigNet behind two seams. To run it over any networking solution you implement two small adapters and reuse one boilerplate driver - FnVoice never opens a socket itself.

your transport  --(opaque bytes)-->  FnVoiceEngine  --(PCM)-->  AudioSource
     ^  IVoiceTransport (you write)        ^  IVoiceSession (you write)
     |                                     |
     +----- FnVoiceHost driver (copy the provided one, boilerplate) -----+

• IVoiceTransport - you send FnVoice's opaque encoded bytes over your transport, and on receive you hand the bytes back with FnVoiceEngine.OnVoicePacket(senderId, bytes).
• IVoiceSession - you tell FnVoice who is in the call (a stable uint peer id per player + join/leave).
• FnVoiceHost - the pipeline glue (encode/decode threads, per-speaker players). This is identical for every transport, so copy the provided LoopbackVoiceDemo (rename it) - you don't have to write it.

Everything above the wire - VAD, WebRTC APM (AEC/AGC/NS), the AudioMixer bus, follow-transform positional voice - keeps working unchanged.

---

The seam interfaces (for reference)

namespace FigNet.Voice
{
    public interface IVoiceTransport
    {
        void Broadcast(ArraySegment<byte> encodedFrame);   // send the frame to everyone (always unreliable)
        int GetRoundTripMs();                              // -1 if unknown; feeds the adaptive jitter buffer
    }

    public interface IVoiceSession
    {
        uint LocalPeerId { get; }
        bool IsActive { get; }
        IReadOnlyList<VoicePeer> Peers { get; }
        bool TryGetPeer(uint peerId, out VoicePeer peer);
        event Action<VoicePeer> PeerJoined;   // -> a decoder + AudioSource is created for the peer
        event Action<uint> PeerLeft;          // -> that peer's decoder + AudioSource is destroyed
        event Action SessionEnded;            // -> encode/decode threads stop
    }

    public readonly struct VoicePeer { public readonly uint Id; public readonly bool IsLocal; public readonly string DisplayName; }
}

You never touch AudioDecodeThread/DecoderState/VoiceChatPacket directly - the FnVoiceHost driver does.

---

Step 1 - implement IVoiceTransport (PUN2 example)

Voice frames are just opaque byte[]. In PUN2 the simplest carrier is PhotonNetwork.RaiseEvent.

using System;
using ExitGames.Client.Photon;
using Photon.Pun;
using Photon.Realtime;
using FigNet.Voice;

public sealed class PunVoiceTransport : IVoiceTransport, IOnEventCallback
{
    private const byte VoiceEventCode = 199;   // any unused Photon event code (0-199)
    private static readonly RaiseEventOptions ToOthers = new RaiseEventOptions { Receivers = ReceiverGroup.Others };

    public PunVoiceTransport()  => PhotonNetwork.AddCallbackTarget(this);
    public void Dispose()       => PhotonNetwork.RemoveCallbackTarget(this);

    // FnVoice -> your transport. Voice is always unreliable (loss is absorbed by PLC/FEC).
    public void Broadcast(ArraySegment<byte> frame)
    {
        PhotonNetwork.RaiseEvent(VoiceEventCode, ToBytes(frame), ToOthers, SendOptions.SendUnreliable);
    }

    public int GetRoundTripMs() => (int)PhotonNetwork.GetPing();

    // your transport -> FnVoice. Photon hands us the sender's ActorNumber; feed it straight in.
    public void OnEvent(EventData e)
    {
        if (e.Code != VoiceEventCode) return;
        var bytes = (byte[])e.CustomData;
        FnVoiceEngine.OnVoicePacket((uint)e.Sender, new ArraySegment<byte>(bytes));
    }

    private static byte[] ToBytes(ArraySegment<byte> f)
        => (f.Offset == 0 && f.Count == f.Array.Length) ? f.Array : f.ToArray();
}

Step 2 - implement IVoiceSession (PUN2 example)

The session is just your player roster, keyed by a stable uint id. In PUN2 that's the ActorNumber.

using System;
using System.Collections.Generic;
using Photon.Pun;
using Photon.Realtime;
using FigNet.Voice;

public sealed class PunVoiceSession : IVoiceSession, IInRoomCallbacks
{
    private readonly List<VoicePeer> _peers = new List<VoicePeer>();

    public PunVoiceSession() => PhotonNetwork.AddCallbackTarget(this);
    public void Dispose()    => PhotonNetwork.RemoveCallbackTarget(this);

    public uint LocalPeerId => (uint)PhotonNetwork.LocalPlayer.ActorNumber;
    public bool IsActive    => PhotonNetwork.InRoom;

    public IReadOnlyList<VoicePeer> Peers
    {
        get
        {
            _peers.Clear();
            foreach (Player p in PhotonNetwork.PlayerList) _peers.Add(ToPeer(p));
            return _peers;
        }
    }

    public bool TryGetPeer(uint id, out VoicePeer peer)
    {
        foreach (Player p in PhotonNetwork.PlayerList)
            if ((uint)p.ActorNumber == id) { peer = ToPeer(p); return true; }
        peer = default; return false;
    }

    public event Action<VoicePeer> PeerJoined;
    public event Action<uint> PeerLeft;
    public event Action SessionEnded;

    private static VoicePeer ToPeer(Player p) => new VoicePeer((uint)p.ActorNumber, p.IsLocal, p.NickName);

    public void OnPlayerEnteredRoom(Player p) => PeerJoined?.Invoke(ToPeer(p));
    public void OnPlayerLeftRoom(Player p)    => PeerLeft?.Invoke((uint)p.ActorNumber);

    // Raise SessionEnded from your matchmaking OnLeftRoom callback (IMatchmakingCallbacks) - call:
    public void NotifyLeftRoom() => SessionEnded?.Invoke();

    // remaining IInRoomCallbacks members - leave empty
    public void OnRoomPropertiesUpdate(ExitGames.Client.Photon.Hashtable p) { }
    public void OnPlayerPropertiesUpdate(Player t, ExitGames.Client.Photon.Hashtable c) { }
    public void OnMasterClientSwitched(Player n) { }
}

Step 3 - the FnVoiceHost driver (copy, don't write)

This is the transport-agnostic pipeline glue. Copy the shipped LoopbackVoiceDemo (Assets/Modules/FnVoice/VoiceChat/Samples/Loopback/) into your project and rename it - it already does exactly this. For reference, the generic form is:

using System;
using MemoryPack;
using UnityEngine;
using FigNet.Voice;

public class FnVoiceHost : MonoBehaviour   // must live in Assembly-CSharp (default Unity scripts)
{
    private AudioDecodeThread _decode;
    private AudioEncodeThread _encode;
    private VoiceChatRecorder _recorder;
    private ushort _seq;

    private void Start()
    {
        // FnVoiceEngine.Configure(...) must already have run (see bootstrap below).
        FnVoiceEngine.PacketSink = OnDecodedPacket;      // where inbound frames are decoded
        _decode = AudioDecodeThread.CreateAndStart(this);

        var s = FnVoiceEngine.Session;
        s.PeerJoined  += OnPeerJoined;
        s.PeerLeft    += OnPeerLeft;
        s.SessionEnded += OnSessionEnded;
        foreach (var p in s.Peers) OnPeerJoined(p);       // spawn players already in the call
    }

    private void OnPeerJoined(VoicePeer peer)
    {
        if (peer.IsLocal)                                 // us: capture + encode, no player
        {
            _recorder = VoiceChatRecorder.Instance;
            if (_recorder != null) _recorder.OnSampleAvailable += OnMic;
            _encode = AudioEncodeThread.CreateAndStart(this);
            _encode.ProcessAudioFrame = OnEncoded;
        }
        else                                              // remote: a player + its decoder
        {
            var go = new GameObject("Voice_" + peer.Id);
            go.transform.SetParent(transform, false);
            var player = go.AddComponent<VoiceChatPlayerOnFilterRead>();   // RequireComponent adds the AudioSource
            var decoder = _decode.AddDecoder(peer.Id);
            player.Init(peer.Id, decoder);
            VoicePlayers.Register(peer.Id, player);        // lets VoiceLevels/roster resolve the speaker
        }
    }

    private void OnPeerLeft(uint id)
    {
        _decode?.RemoveDecoder(id);
        if (VoicePlayers.TryGet(id, out var p) && p != null) Destroy(p.transform.gameObject);
        VoicePlayers.Unregister(id);
    }

    private void OnSessionEnded()
    {
        _encode?.Dispose(); _encode = null;
        _decode?.Dispose(); _decode = null;
        VoicePlayers.Clear();
    }

    private void OnMic(float[] pcm) => _encode?.AddAudioFrame(pcm);

    // encode thread: compress -> serialize -> hand the opaque bytes to YOUR transport
    private void OnEncoded(float[] pcm)
    {
        var packet = VoiceChatUtils.Compress(pcm);
        packet.PacketId = ++_seq;
        var bytes = MemoryPackSerializer.Serialize(new VoipData { voiceChatPacket = packet });
        FnVoiceEngine.Transport.Broadcast(new ArraySegment<byte>(bytes));
    }

    // inbound: route the decoded packet to the sender's decoder
    private void OnDecodedPacket(uint senderId, VoiceChatPacket packet)
    {
        _decode?.AddCompressedAudio(senderId, packet);
        if (VoicePlayers.TryGet(senderId, out var p) && p != null) p.OnAudioFrameProcess();
    }

    private void OnDestroy()
    {
        if (_recorder != null) _recorder.OnSampleAvailable -= OnMic;
        FnVoiceEngine.PacketSink = null;
        OnSessionEnded();
    }
}

Wiring it up

After you have joined your room / session:

FnVoiceEngine.Configure(new PunVoiceTransport(), new PunVoiceSession());
gameObject.AddComponent<FnVoiceHost>();   // or have it already in the scene

Scene needs: the FnVoice Resources (FnVoiceAudioSettings) present, and a VoiceChatRecorder (Auto-Start-On-Start, a mic device selected). No VoipConnectionManager / FigNet networking.

---

Fusion 2 (and everything else)

Identical shape - only the send primitive changes:

• IVoiceTransport.Broadcast -> a Fusion RPC / message to the other clients over an unreliable channel (voice is high-rate + loss-tolerant).
• receive -> your RPC/message handler calls FnVoiceEngine.OnVoicePacket(senderId, bytes).
• IVoiceSession -> project the NetworkRunner player list to VoicePeer (use PlayerRef.PlayerId or RawEncoded as the uint), and raise PeerJoined/PeerLeft from INetworkRunnerCallbacks.OnPlayerJoined/ OnPlayerLeft.

Notes / caveats

• The bytes are opaque. Carry them verbatim; don't inspect or re-encode. FnVoice owns the frame format.
• senderId must be a stable per-peer uint and must match on both ends (Photon ActorNumber, Fusion PlayerId, ...). The decoder is keyed by it; a mismatch silently drops that peer's audio.
• Unreliable is right. PLC/FEC + the adaptive jitter buffer handle loss; a reliable channel adds latency.
• RTT is optional. Return -1 from GetRoundTripMs() if your transport can't supply it - the jitter buffer falls back gracefully.
• Encryption is your transport's responsibility now (PUN2/Fusion bring their own DTLS/TLS). FnVoice does not encrypt the frame.
• No FigNet/Entangle needed on this path. The FnVoiceFenceGuardTests test keeps the audio core provably free of FigNet networking, which is what makes this possible.
• Same-assembly note: FnVoiceHost sets FnVoiceEngine.PacketSink (internal); keep it in the default Assembly-CSharp (as normal Unity scripts are), or the copied LoopbackVoiceDemo, so it compiles.