View Source Creating a file archive
Mix.install(
[
{:lastfm_archive,
git: "https://github.com/boonious/lastfm_archive.git", branch: "test-data-factory"}
],
config: [
lastfm_archive: [
data_dir: "./lastfm_data/",
user: ""
]
]
)
alias LastfmArchive.Livebook, as: LFM_LB
:ok
Introduction
This guide uses the lastfm_archive library to create an archive containing scrobbles: music tracks that Lastfm users have been listening to. It downloads scrobbles via the Lastfm API and stores them as raw JSON data in a local file location.
Requirement
- install and start Livebook
- run this Livebook guide
- configure Livebook as instructued below, click on
Notebook dependencies and setup
,Setup
(above)
Configuration
Lastfm archive has been configured as a dependency in Setup
above. You need to check and modify the following configs:
user
: specify a Lastfm username in this config or share it via aLFM_USER
secret (see Secrets management)data_dir
(optional): by default scrobbles data is stored in~/lastfm_data/
directory within your home directory. Modifiy this location if other directory is preferred
API key environment variable
To access the API provided by Lastfm, you need to apply for a 32-digit string key. This key will be used when downloading data via Lastfm API.
Once you have obtained this key, add it as LFM_API_KEY
secret and share it with this Livebook - see Secrets management for further details.
Scope
Run the following code (click Evaluate
below) to find out what are you about to archive:
LFM_LB.info()
Archiving
Run the follow to begin archiving. This will initate a process that fetches scrobbles on a daily basis, at around every 1s (within permissable Lastfm API request rate). The process is memoised or cached. It can be halted (click Stop
) and restarted without data being re-fetched and the API being unnecessarily called again.
LastfmArchive.sync()
Year option
:year
: archiving scrobbles from a given year option. For example:
# change year to one containing scrobbles
LastfmArchive.default_user() |> LastfmArchive.sync(year: 2023)
Date option
:date
archiving scrobbles from a given date
# change date to one containing scrobbles
LastfmArchive.default_user() |> LastfmArchive.sync(date: ~D[2023-09-12], overwrite: true)
# a = [~D[2007-10-13], ~D[2008-04-05], ~D[2011-12-23], ~D[2011-05-01], ~D[2012-02-10],
# ~D[2012-03-01], ~D[2017-10-17], ~D[2017-11-21], ~D[2018-10-04], ~D[2023-09-12]]
# a |> Enum.each(fn d -> LastfmArchive.sync("boonlow", date: d, overwrite: true) end)
Overwrite option
:overwrite
any existing scrobbles, ignoring archiving status cache
LastfmArchive.default_user() |> LastfmArchive.sync(date: ~D[2023-09-03], overwrite: true)
Status
Run the following for a heatmap and counts, checking the archiving progress. Rerun the code to get the latest status.
[LFM_LB.monthly_playcounts_heatmap(), LFM_LB.yearly_playcounts_table()]
|> Kino.Layout.grid(columns: 2)