## Link to GitHub Issue or related Pull Request, if one exists
#0
## Description of change
MSE method of streaming video needs the dimensions up front. This new
api delivers that. Also provides an easier way to determine which
screens are available and a way to discover port number.
## Testing
See pending changes in substream project
## Link to GitHub Issue or related Pull Request, if one exists
fixes#875
## Description of change
Adds `-apistream`, an optional HTTP video stream of the mirrored screen.
It listens on the API port +2.
Two endpoints, sharing the same `screen`, `fps` and `q` parameters:
/stream.mjpg JPEG frames, for clients with no container support
/stream.h264 H.264 annex-b, for an app driving MediaCodec or
VideoToolbox itself
One encoder per connection, fed by a per-screen pump that always hands
over the newest frame, so a slow reader drops frames instead of building
a backlog. `capture.get_jpg` behaviour is unchanged.
Additional documentation for developers:
https://github.com/spice2x/spice2x.github.io/wiki/Video-Stream
## Testing
## Link to GitHub Issue or related Pull Request, if one exists
#0
## Description of change
Significantly speeds up API screen capture and D3D9 screenshots saving.
Two reasons for doing this:
1. We now have a 4K game (GITADORA) and existing capture code was taking
multiple seconds.
2. Renewed user interest on streaming as we have a couple more companion
apps in active development.
**API screen capture (streaming), 1280x720:** 14.3ms -> 6.3ms per frame.
Back buffer copies go to pooled `D3DPOOL_SYSTEMMEM` surfaces via
`GetRenderTargetData` instead of allocating a lockable render target
every frame, and TooJpeg is replaced with libjpeg-turbo (encode 9.8ms ->
3.0ms). MSAA remains unsupported
**Screenshots for GITADORA arena model, across 4 screens with one of
them 4K**: 4068ms -> 124ms. `D3DXSaveSurfaceToFileA` is replaced with
fpng (encode 4043ms -> 76ms) and the screens encode in parallel.
Dropping D3DX also removes the `d3dx9_43.dll` ... `d3dx9_24.dll` probing
loop, so screenshots no longer fail outright on machines with no D3DX9
runtime installed.
Screenshot surfaces are read on the present thread, so no D3D call
reaches another thread for screenshots. This fixes a hang in DDR X2
introduced earlier in the branch: its device has no internal locking,
and reading the surface on a pool thread while the present thread sat
inside `GetRenderTargetData` left the game's own render thread
deadlocked.
## Testing
- **GITADORA** (arena model, D3D9Ex, 4K main plus three subscreens,
windowed) with
`-screenshotsub`: three sets of four screenshots, images verified
correct. Completion
order differs between sets, so the screens really are encoding in
parallel.
- **LovePlus** (KLP, plain D3D9, 768x1360): covers the inline path used
by games whose
image processing must not leave the present thread.
- **API screen capture** through a companion app: live video correct
throughout.
- **Print Screen** bound as the screenshot key: the clipboard copy
succeeded on every shot.
- Quitting the game after capturing leaves no `IDirect3DDevice9`
reference count warning,
so the pooled readback surfaces are released along with the device.
Add a timeout and skip signalling to the D3D9 capture path so the API
thread no longer waits indefinitely when Present stops during loading.
Return the last successful JPEG frame as a fallback.
## Link to GitHub Issue or related Pull Request, if one exists
#746
## Description of change
* Fix `capture.get_jpg` blocking indefinitely when D3D9 screen capture
cannot
complete during game loading (e.g. no Present calls).
* Add a 2-second timeout to `graphics_capture_receive_jpeg()` and cancel
pending capture requests on timeout.
* Signal capture skip from D3D9 failure paths via
`graphics_capture_skip()`.
* Cache the last successful JPEG per screen and return it as a fallback
when a new capture fails.
Tested with **stock iOS Spice Companion** only. I have not tested
Android or
other Companion clients. The iOS client implementation differs from the
others; this fix is server-side only.
No config file, CLI option, or API schema changes.
## Testing
* Built locally with llvm-mingw cross-compiler (WSL).
* Docker build (`src/spice2x/build_docker.sh`): Pending
* Tested with stock iOS Spice Companion on iPad: Mirror remains
connected
during game loading instead of returning to KeyPad.
## Link to GitHub Issue, if one exists
n/a
## Description of change
Allow the server to override client's quality options for screen
mirroring.
On Windows / Android versions (flutter-based) clients, this is already
in the options of those apps. However, the iOS version lacks the quality
options.
## Testing
Tested with iOS and Windows clients.
bicarus