Full-text search and lyrics in a Go music server
- 7 minutes read - 1375 wordsThe Tunes server started as a way to stream my music library from a Raspberry Pi. It’s accumulated features since then. Two recent additions changed how I interact with it day-to-day: accent-insensitive full-text search across the entire library, and lyrics, both embedded from audio files and fetched from LRClib.
The search problem
Searching a music library is harder than it sounds. You want “bjork” to match “Björk”. You want “cafe” to match “Café”. You want “creme” to match “Crème Brûlée” in an album title. And you want it fast enough that results appear as you type.
The previous search was a LIKE query with wildcards. It worked, but it couldn’t handle diacritics and got slow on a 15,000-track library with multi-column matches.
FTS5 with unicode61
SQLite’s FTS5 extension does exactly what we need. The unicode61 tokenizer with remove_diacritics 2 strips all Unicode diacritical marks during indexing and querying:
func (s *SearchFTS) InitializeFTS() error {
createSQL := `
CREATE VIRTUAL TABLE IF NOT EXISTS tracks_fts USING fts5(
track_id UNINDEXED,
name,
artist,
album_artist,
album,
composer,
genre,
tokenize='unicode61 remove_diacritics 2'
)`
if err := s.db.Exec(createSQL).Error; err != nil {
if strings.Contains(err.Error(), "no such module: fts5") {
Sugar.Warn("FTS5 module not available - search will be disabled.")
s.fts5Available = false
return nil
}
return fmt.Errorf("failed to create FTS5 table: %w", err)
}
s.fts5Available = true
return nil
}track_id UNINDEXED stores the ID in the FTS table for joins but doesn’t tokenize it. If FTS5 isn’t compiled into the SQLite build, search degrades gracefully instead of crashing.
The index rebuilds atomically in a transaction: delete all FTS data, repopulate from the tracks table using COALESCE for NULL safety:
func (s *SearchFTS) RebuildIndex(ctx context.Context) error {
return s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
if err := tx.Exec("DELETE FROM tracks_fts").Error; err != nil {
return fmt.Errorf("failed to clear FTS table: %w", err)
}
populateSQL := `
INSERT INTO tracks_fts (track_id, name, artist, album_artist,
album, composer, genre)
SELECT track_id, COALESCE(name, ''), COALESCE(artist, ''),
COALESCE(album_artist, ''), COALESCE(album, ''),
COALESCE(composer, ''), COALESCE(genre, '')
FROM i_tunes_tracks WHERE deleted_at IS NULL`
return tx.Exec(populateSQL).Error
})
}Query building
User queries are split into words, each gets a * suffix for prefix matching, and they’re joined with implicit AND. FTS5 special characters are stripped to prevent injection:
func buildFTSQuery(query string) string {
words := strings.Fields(strings.TrimSpace(query))
var parts []string
for _, word := range words {
escaped := escapeFTSToken(word)
if escaped != "" {
parts = append(parts, escaped+"*")
}
}
return strings.Join(parts, " ")
}
func escapeFTSToken(token string) string {
var result strings.Builder
for _, r := range token {
switch r {
case '"', '*', '^', '(', ')', '{', '}', '[', ']', ':', '-':
continue
default:
result.WriteRune(r)
}
}
return result.String()
}Typing “aphex win” becomes aphex* win*, which matches “Aphex Twin” via prefix expansion. Typing “cafe del” matches “Café Del Mar” because the tokenizer already stripped the accent at index time.
Unified search endpoint
The /search?q= endpoint returns artists, albums, and tracks in a single response, each ranked independently by BM25:
type UnifiedSearchResponse struct {
Artists []ArtistInfo `json:"artists"`
Albums []AlbumInfo `json:"albums"`
Tracks []ITunesTrack `json:"tracks"`
}
func (ls *LibraryService) HandleSearch(w http.ResponseWriter, r *http.Request) {
query := strings.TrimSpace(r.URL.Query().Get("q"))
artists := ls.searchArtistsFTS(ctx, repo, query, 5)
albums := ls.searchAlbumsFTS(ctx, repo, query, 10)
tracks := ls.searchTracksFTS(ctx, repo, query, 30)
// ...
}Each section queries specific FTS5 columns. Artist search hits both artist and album_artist, then merges and aggregates. The BM25 scoring comes from FTS5’s built-in function:
SELECT track_id, bm25(tracks_fts) as rank
FROM tracks_fts
WHERE tracks_fts MATCH ?
ORDER BY rank
LIMIT ?BM25 order is preserved through joins by building a track map and reordering after the GORM query:
trackMap := make(map[int]ITunesTrack, len(tracks))
for _, t := range tracks {
if t.TrackID != nil {
trackMap[*t.TrackID] = t
}
}
ordered := make([]ITunesTrack, 0, len(trackIDs))
for _, id := range trackIDs {
if t, ok := trackMap[id]; ok {
ordered = append(ordered, t)
}
}Album search uses a two-stage approach: FTS returns track IDs, those map to album_key, then the album_cache table provides pre-aggregated album metadata. This avoids re-aggregating album info on every search.
Lyrics from audio files
The music scanner already walks the filesystem to index tracks using github.com/dhowden/tag. Adding lyrics extraction was a matter of probing the right metadata keys:
func (ms *MusicScanner) extractLyricsFromMetadata(rawMetadata any) *ExtractedLyrics {
var lyrics ExtractedLyrics
switch raw := rawMetadata.(type) {
case map[string]any:
for _, key := range []string{
"lyrics", "USLT", "unsynchronised_lyrics",
"unsynced_lyrics", "LYRICS", "\xa9lyr",
} {
if val, ok := raw[key]; ok {
switch v := val.(type) {
case string:
if v != "" { lyrics.PlainLyrics = v }
case []byte:
if len(v) > 0 { lyrics.PlainLyrics = string(v) }
}
}
}
// Also check SYLT for synchronized lyrics
for _, key := range []string{"SYLT", "synchronised_lyrics", "synced_lyrics"} {
// ... same pattern
}
}
return &lyrics
}USLT is the ID3v2 unsynchronized lyrics frame, "\xa9lyr" is the M4A/AAC lyrics atom, SYLT is synchronized (timestamped) lyrics. The scanner handles both plain text and timestamped LRC format.
LRClib integration
For tracks without embedded lyrics, the server fetches from LRClib, a free, open lyrics database with synced timestamps.
The lyrics service uses golang.org/x/time/rate to respect API limits:
const (
lrcLibBaseURL = "https://lrclib.net/api"
lrcLibUserAgent = "Tunes/1.0 (https://github.com/manz/tunes-server)"
lrcLibRateLimit = 2
)
func NewLyricsService(db *gorm.DB) *LyricsService {
return &LyricsService{
repo: NewLyricsRepository(db),
httpClient: &http.Client{Timeout: 15 * time.Second},
rateLimiter: rate.NewLimiter(rate.Limit(lrcLibRateLimit), 1),
}
}When fetching, the service validates that the returned lyrics match the track’s duration within 5 seconds. This prevents mismatches: same song title by different artists, live vs studio versions:
if duration > 0 && lrcResponse.Duration > 0 {
if math.Abs(float64(duration)-lrcResponse.Duration) > durationTolerance {
return nil, ErrLyricsNotFound
}
}The handler implements fetch-through caching: check the database first, if miss then fetch from LRClib, save, and return. No external call on subsequent requests for the same track.
Lyrics are stored with GORM’s FirstOrCreate + Assign for clean upserts:
func (r *LyricsRepository) SaveLyrics(ctx context.Context, lyrics *TrackLyrics) error {
return r.db.WithContext(ctx).
Where(TrackLyrics{TrackID: lyrics.TrackID}).
Assign(TrackLyrics{
LRCLibID: lyrics.LRCLibID,
Instrumental: lyrics.Instrumental,
PlainLyrics: lyrics.PlainLyrics,
SyncedLyrics: lyrics.SyncedLyrics,
Source: lyrics.Source,
}).
FirstOrCreate(lyrics).Error
}Background enrichment
A POST /lyrics/enrich endpoint triggers a goroutine that processes tracks without lyrics in batches of 50. It finds un-enriched tracks with a LEFT JOIN:
err := r.db.Joins(
"LEFT JOIN track_lyrics ON i_tunes_tracks.track_id = track_lyrics.track_id",
).Where("track_lyrics.id IS NULL").
Where("i_tunes_tracks.deleted_at IS NULL").
Limit(limit).
Find(&tracks).ErrorRate-limited at 2 requests per second, a 15,000-track library takes a few hours to fully enrich. The mutex prevents concurrent enrichment runs:
func (s *LyricsService) StartBackgroundEnrichment(ctx context.Context, ...) {
s.mu.Lock()
if s.isEnriching {
s.mu.Unlock()
return
}
s.isEnriching = true
s.mu.Unlock()
go func() {
defer func() { s.mu.Lock(); s.isEnriching = false; s.mu.Unlock() }()
tracks, _ := s.repo.GetTracksWithoutLyrics(ctx, 50)
// ... process batch, respect rate limiter
}()
}Admin via sudoers
The authorization model is Unix-inspired. Instead of a boolean is_admin on the User model, there’s a dedicated sudoers table with audit trail:
type Sudoer struct {
ID int `gorm:"primaryKey;autoIncrement"`
UserID int `gorm:"column:user_id;uniqueIndex;not null"`
GrantedBy int `gorm:"column:granted_by;not null"`
GrantedAt time.Time `gorm:"autoCreateTime"`
ExpiresAt *time.Time `gorm:"column:expires_at"`
Active bool `gorm:"column:active;default:true"`
}Destructive operations (deleting tracks, managing users) additionally require a sudo token, obtained by re-authenticating via WebAuthn passkey. Two-step elevation: request a challenge, prove you’re you, get a time-limited token. This prevents a stolen session from doing real damage.
Routes use higher-order functions for auth gating:
func sudoerRoute(authMiddleware func(http.Handler) http.Handler,
authService *AuthService) func(http.HandlerFunc) http.Handler {
return func(handler http.HandlerFunc) http.Handler {
return authMiddleware(requireSudoerMiddleware(authService)(handler))
}
}The first user must be created via CLI before the server starts: tunes-server users add <username> --sudoer. No web-based registration for the initial admin. You need shell access.
What changed
The server went from “stream my library” to “actually find what I want to play.” FTS5 with accent stripping makes search work the way you’d expect: type a rough approximation of the artist name, get the right results. Lyrics with synced timestamps let the iOS app show karaoke-style scrolling text. Background enrichment means I didn’t have to manually tag anything. It just filled in over a couple of evenings.
Still no replacement for Apple Music’s catalog. But for my 15,000 tracks, it works better than iTunes Match ever did.