Video

ui.Video shows a video that plays: a trailer with sound and controls, a clip looping behind the content, as a launcher's library or a music player's hero shows one, or a preview in a grid. It shows a ui.VideoPlayer, and is laid out, fitted and rounded as an Image is.

A player comes from one of two places:

• ui/video opens video files and streams (MP4 and MOV with H.264 and AAC everywhere, more where the system has the codecs, HLS streams on macOS, Windows and browsers) with the system's own media framework, which decodes them in hardware and plays their sound. Its pictures reach the window's renderer where the decoder left them, on the GPU.
• A decoder in Go: ui.NewGIFDecoder and ui.NewAPNGDecoder play animated GIFs and PNGs on every system, without sound, as backgrounds that should never depend on the system; a ui.VideoDecoder of your own plays another format.

import "github.com/egoist/mygo/ui/video"

//go:embed trailer.mp4
var trailerMP4 []byte

app.trailer, err = video.OpenData(trailerMP4) // or video.Open(path), video.OpenURL(url)
if err != nil {
	log.Fatal(err)
}

// In the view:
ui.Video(c, app.trailer).Fill().Fit(ui.Contain)

//go:embed waves.gif
var wavesGIF []byte

d, err := ui.NewGIFDecoder(wavesGIF)
if err != nil {
	log.Fatal(err)
}
app.waves, err = ui.NewVideoPlayer(d)
if err != nil {
	log.Fatal(err)
}
app.waves.SetLoop(true).SetMuted(true)

// In the view:
ui.Video(c, app.waves).Fill().Fit(ui.Cover)

The video example plays a GIF behind a window's content and a trailer with controls over it.

Players

Make players once, not in the view, as bitmaps. A new player plays as soon as a Video shows it. Pause holds it at the frame it shows, and Play goes on from there, or plays it again from its start once it ended; Playing tells which, and Ended whether it played to its end, where it shows its last frame. SetLoop plays it again each time it ends, without a gap. Seek shows the frame at a time and plays on from there, unless paused; Position and Duration tell where it plays and how long it is (0 for a live stream). SetRate plays it faster or slower, its sound at the same pitch; SetVolume and SetMuted set its sound. Err tells what stopped it before its end, as a file that does not decode or a stream that broke off. Close stops it, and its sound. Call a player's methods on the main thread, in the view or an input handler, and show it in one Video at a time.

label := "Pause"
if !p.Playing() {
	label = "Play"
}
if ui.Button(c.Key("play"), label).Clicked() {
	if p.Playing() {
		p.Pause()
	} else {
		p.Play()
	}
}
pos := p.Position().Seconds()
if ui.Slider(c.Key("seek"), &pos, 0, p.Duration().Seconds()).Label("Position").Changed() {
	p.Seek(time.Duration(pos * float64(time.Second)))
}

ui.NewVideoPlayer decodes the first frame at once, which Video shows from its first frame and which reports a video that does not decode. A system's video opens at once and loads in the background: Ready is false and Size 0×0 until its first picture came, and Buffering is true while a stream waits for data, for the view to show a spinner. Each change of these builds a frame of the window, so the view never polls.

Playing

A player decodes its frames on a goroutine of its own, a couple ahead of the one showing, into frames it uses again, and each frame reaches the screen through a bitmap's Update as it comes due: the window draws one frame for each of the video's, which After asks for, and none between. When the decoder falls behind, the window waits for it without drawing, and of the frames that came due meanwhile shows only the last, so the video keeps to the clock rather than lag behind it.

A system's video keeps the clock its sound plays by. As each frame of the window builds, the player takes the picture due as the frame shows, where the decoder left it, and the window draws again as the next comes due: a 24 fps video on a 120 Hz display draws 24 frames a second.

A muted video is a picture, as a background is. It moves on only while the view builds it and some of the window shows: one that went out of the view, as on another tab, or in a window that is minimized, hidden or covered by other windows (on macOS, and in browsers), goes on from where it stood once it shows again, and decodes nothing meanwhile. While the desktop asks for less motion (Preferences.ReduceMotion), it stands still: a decoder's at its first frame. A video with sound plays on out of sight, as a browser tab's does, and plays under less motion too, as the user asked for it; a decoder's video has no sound, and is always the former kind.

Decoders

ui.NewGIFDecoder decodes animated GIFs, in Go, as browsers show them: each frame drawn over what the ones before it left, as its disposal says, and a frame of 10 ms or less shown for 100 ms. The GIF decodes into memory at once, a byte a pixel for each frame, and each frame is drawn as it plays. It plays once or loops as SetLoop says, whatever loop count the GIF gives.

A ui.VideoDecoder of your own plays another format: Next returns the next frame, a picture of any image type and how long it shows, or io.EOF after the last, and Rewind goes back to before the first. The player copies each picture, so a decoder may draw every frame into the same one, and calls the decoder from one goroutine at a time. A live stream that an app decodes itself, as with FFmpeg, reaches the screen through a bitmap's UpdateYUV instead, its frames in Y′CbCr as the decoder gives them, which the GPU converts.

type frames struct {
	pics []*image.RGBA
	next int
}

func (f *frames) Next() (ui.VideoFrame, error) {
	if f.next == len(f.pics) {
		return ui.VideoFrame{}, io.EOF
	}
	f.next++
	return ui.VideoFrame{Image: f.pics[f.next-1], Duration: time.Second / 30}, nil
}

func (f *frames) Rewind() error { f.next = 0; return nil }

ui.NewAPNGDecoder decodes animated PNGs the same way, each frame drawn as its disposal and blending say, and keeps each frame compressed until it plays. A PNG that is not animated is a video of one frame.

A decoder that also has a method Duration() time.Duration reports the video's length through Duration; GIFs' and APNGs' do. Seek on a decoder's video decodes from its start up to the frame.

System video

Package ui/video plays with the system's media framework. Apps that do not import it link none of it.

video.Open opens a file, video.OpenURL a file:, http: or https: URL or an HLS playlist, and video.OpenData bytes in memory, as an embedded file, without writing them to a file. video.ErrUnsupported reports a system without a framework that plays video, as a Linux without GStreamer and its base plugins or a Windows Server without its Media Foundation feature, for the app to show a GIF instead. What formats play is the system's: H.264 and AAC in MP4 play on all of them. Linux needs gstreamer1.0-plugins-base, and gstreamer1.0-libav (or another H.264 decoder) for H.264; a missing decoder shows as Err.

Windows on Arm's MPEG-4 reader ignores a track's edit list. An H.264 video with B-frames (x264's default) has one that takes back its reordering delay, so there each picture shows that much late, up to a few frames behind its sound. Encode a clip without B-frames (-x264-params bframes=0) where that matters, as the example's trailer is.

In browsers a video with sound may not play before the user interacted with the page: it then plays muted, as the browser allows.

Accessibility

Videos are decorations that assistive technology does not see, unless Label names them, as images are.