0 Fork 0

mirror of https://github.com/langchain-ai/langchain.git synced 2026-10-07 10:25:17 +03:00

Files

T

Matt Robinson 07a407d89a feat: adds UnstructuredURLLoader for loading data from urls (#979 )

### Summary

Adds a `UnstructuredURLLoader` that supports loading data from a list of
URLs.


### Testing

```python
from langchain.document_loaders import UnstructuredURLLoader

urls = [
    "https://www.understandingwar.org/backgrounder/russian-offensive-campaign-assessment-february-8-2023",
    "https://www.understandingwar.org/backgrounder/russian-offensive-campaign-assessment-february-9-2023"
]
loader = UnstructuredURLLoader(urls=urls)
raw_documents = loader.load()
```

2023-02-10 10:18:38 -08:00

1.6 KiB

Raw Blame History

URL

This covers how to load HTML documents from a list of URLs into a document format that we can use downstream.

In [1]:

 from langchain.document_loaders import UnstructuredURLLoader

In [2]:

urls = [
    "https://www.understandingwar.org/backgrounder/russian-offensive-campaign-assessment-february-8-2023",
    "https://www.understandingwar.org/backgrounder/russian-offensive-campaign-assessment-february-9-2023"
]

In [3]:

loader = UnstructuredURLLoader(urls=urls)

In [4]:

data = loader.load()

1.6 KiB Raw Blame History

URL

1.6 KiB

Raw Blame History