Scidata.Squad (Scidata v0.1.5) View Source

Module for downloading the SQuAD1.1 dataset.

Link to this section Summary

Functions

Downloads the SQuAD training dataset

Downloads the SQuAD test dataset

Convert result of download/0 or download_test/0 to map for use with Explorer.DataFrame.

Link to this section Functions

Downloads the SQuAD training dataset

Examples

iex> Scidata.Squad.download()
[
  %{
    "paragraphs" => [
      %{
        "context" => "Architecturally, the school has a...",
        "qas" => [
          %{
            "answers" => [%{"answer_start" => 515, "text" => "Saint Bernadette Soubirous"}],
            "id" => "5733be284776f41900661182",
            "question" => "To whom did the..."
          }, ...
        ]
      }
    ],
    "title" => "University_of_Notre_Dame"
  }, ...
]

Downloads the SQuAD test dataset

Examples

iex> Scidata.Squad.download_test()
[
  %{
    "paragraphs" => [
      %{
        "context" => "Super Bowl 50 was an American football game t...",
        "qas" => [
          %{
            "answers" => [
              %{"answer_start" => 177, "text" => "Denver Broncos"},...
            ],
            "id" => "56be4db0acb8001400a502ec",
            "question" => "Which NFL team represented the AFC at Super Bowl 50?"
          },
        ]
      }
    ],
    "title" => "Super_Bowl_50"
  }, ...
]

Convert result of download/0 or download_test/0 to map for use with Explorer.DataFrame.

Examples

iex> columns_for_df = Scidata.Squad.download() |> Scidata.Squad.to_columns()
%{
  "answer_start" => [515, ...],
  "context" => ["Architecturally, the...", ...],
  "id" => ["5733be284776f41900661182", ...],
  "question" => ["To whom did the Vir...", ...],
  "answer_text" => ["Saint Bernadette Soubirous", ...],
  "title" => ["University_of_Notre_Dame", ...]
}
iex> Explorer.DataFrame.from_map(columns_for_df)
#Explorer.DataFrame<
[rows: 87599, columns: 6]
...
>