PrismCore / Guides

Cached audio and thumbnails

Read already-produced audio and pictures from a playing session — for music recognition or a scrub preview — without fetching, seeking or disturbing playback.

A remux session keeps the segments it has produced on disk (within segmentCacheBytes). Three calls read from that store without ever asking the source for data:

APIReturns
residentRangesThe video spans already on disk, as [ResidentRange] (source seconds)
cachedThumbnail(at:maxDimension:)The opening picture of the resident segment covering a time, as CGImage?
cachedAudio(from:duration:renditionName:)Resident audio decoded to PCM, as CachedAudioClip?

All three use source timestamps, including a non-zero source origin where present — map them onto your player's timeline. residentRanges is not AVPlayerItem.loadedTimeRanges, and a resident range is not a promise that a seek has no latency.

Cached audio

Added in 3.5.2 for music recognition on the remux path:

Swift
let clip = try await session.cachedAudio(from: sourceSeconds - 10, duration: 10)
if let clip {
    // 48 kHz mono Float32: non-LFE channels averaged, LFE dropped
    print(clip.startSeconds, clip.sampleRate, clip.samples.count)
}
  • It never touches playback. It never opens the source, never reports a fetch to the demand logic, never seeks or re-anchors the producer, and never arms a lazy rendition — so it is safe to call while the title plays. (A loopback fetch of an old segment is taken as the playhead, which is why a recogniser must not read segments over HTTP itself.)
  • nil means not resident. Any part of the interval not on disk, a call before start() or after stop(), and segments superseded by an audio-delay change all return nil.
  • What was heard. Times are on the residentRanges axis with the audio offset applied, so a clip is what the viewer heard at that time.
  • Bounded. duration is clamped to CachedAudioClip.maximumDuration (30 s).
  • Which track. renditionName is the master playlist's NAME of the rendition to read; nil means the DEFAULT rendition (or the variant's own track in the muxed shape). A name the master does not declare throws PrismCoreSession.CachedAudioError.unknownRendition rather than reporting "not resident" forever.

The decode runs on a decoder of its own, off the session's actor and the producer's thread. Files are opened under the same eviction lock cachedThumbnail uses, so an eviction cannot take the bytes away mid-read.

Cached thumbnails

Swift
if let image = try await session.cachedThumbnail(at: sourceSeconds, maxDimension: 320) {
    print(image.width, image.height)
}

The image is the segment's opening picture, not the exact frame at seconds. maxDimension is clamped to 32…2048. A miss returns nil without contacting the source; so do fragments larger than 64 MiB. Images are cached (at most 32 images / 16 MiB).

For a preview at any position, including parts not produced yet (it reads the source on its own context), use SeekPreviewService instead.