lukeed

elixir-superscrape

Read an Atom/RSS feed and concatenate it with deduped historical items from past versions saved in the Wayback Machine. (View Docs)

#Install

  1. Add superscrape to your list of dependencies in mix.exs:
def deps do
  [{:superscrape, "~> 0.1.0"}]
end
  1. Ensure superscrape is started before your application:
def application do
  [applications: [:superscrape]]
end

#Usage

View the online documentation for more information.

url = "https://elixir-jobs.org/feed"

SuperScrape.process url
#=> [%{...}, %{...}, %{...}]

SuperScrape.process! url
#=> ["https://elixir-jobs.org/jobs/26", "..."]

SuperScrape.process! [url, "..."]
#=> ["https://elixir-jobs.org/jobs/26", "..."]

SuperScrape.lookup url
#=> ["http://web.archive.org/web/...", "..."]

SuperScrape.open "https://elixir-jobs.org/jobs/26"
#=> %{description: "...", title: "...", ...}

#License

MIT © Luke Edwards

See also

elixir-digitalocean(3), elixir-wayback(3)