Théo LEBRUN cf94091cd0 community[patch]: Skip nested directories when using S3DirectoryLoader (#17829) ...

- **Description:** `S3DirectoryLoader` is failing if prefix is a folder
(ex: `my_folder/`) because `S3FileLoader` will try to load that folder
and will fail. This PR skip nested directories so prefix can be set to
folder instead of `my_folder/files_prefix`.
- **Issue:**
  - #11917
  - #6535
  - #4326
- **Dependencies:** none
- **Twitter handle:** @Falydoor


- [x] **Add tests and docs**: If you're adding a new integration, please
include
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in
`docs/docs/integrations` directory.


- [x] **Lint and test**: Run `make format`, `make lint` and `make test`
from the root of the package(s) you've modified. See contribution
guidelines for more: https://python.langchain.com/docs/contributing/

2024-03-08 16:50:58 -08:00

..

blob_loaders

core: Move document loader interfaces to core (#17723)

2024-03-06 13:59:00 -05:00

parsers

text-splitters[minor], langchain[minor], community[patch], templates, docs: langchain-text-splitters 0.0.1 (#18346)

2024-02-29 18:33:21 -08:00

__init__.py

Merge pull request #18421

2024-03-06 13:16:05 -05:00

acreom.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

airbyte_json.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

airbyte.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

airtable.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

apify_dataset.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

arcgis_loader.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

arxiv.py

community[minor]: Implement lazy_load() for ArxivLoader (#18664)

2024-03-06 09:16:49 -05:00

assemblyai.py

Merge pull request #18421

2024-03-06 13:16:05 -05:00

astradb.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

async_html.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

athena.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

azlyrics.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

azure_ai_data.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

azure_blob_storage_container.py

community[patch]: type ignore fixes (#18395)

2024-03-01 11:21:02 -08:00

azure_blob_storage_file.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

baiducloud_bos_directory.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

baiducloud_bos_file.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

base_o365.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

base.py

core: Move document loader interfaces to core (#17723)

2024-03-06 13:59:00 -05:00

bibtex.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

bigquery.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

bilibili.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

blackboard.py

community[patch]: type ignore fixes (#18395)

2024-03-01 11:21:02 -08:00

blockchain.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

brave_search.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

browserless.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

cassandra.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

chatgpt.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

chm.py

community[patch]: docstrings (#16810)

2024-02-09 12:48:57 -08:00

chromium.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

college_confidential.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

concurrent.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

confluence.py

Merge pull request #18436

2024-03-06 13:15:24 -05:00

conllu.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

couchbase.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

csv_loader.py

community[patch]: Implement lazy_load() for CSVLoader (#18391)

2024-03-01 11:17:08 -08:00

cube_semantic.py

community[patch]: Implement lazy_load() for CubeSemanticLoader (#18535)

2024-03-05 17:32:31 -08:00

datadog_logs.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

dataframe.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

diffbot.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

directory.py

community[minor]: add exclude parameter to DirectoryLoader (#17316)

2024-02-16 09:42:42 -05:00

discord.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

doc_intelligence.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

docugami.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

docusaurus.py

docs: docstrings langchain_community update (#14889)

2023-12-19 08:58:24 -05:00

dropbox.py

infra: add print rule to ruff (#16221)

2024-02-09 16:13:30 -08:00

duckdb_loader.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

email.py

community[minor]: Implement lazy_load() for OutlookMessageLoader (#18668)

2024-03-06 09:15:57 -05:00

epub.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

etherscan.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

evernote.py

community[patch]: Implement lazy_load() for EverNoteLoader (#18538)

2024-03-05 17:29:52 -08:00

excel.py

Docs: fix excel document loader typo (#15470)

2024-01-07 09:33:35 -08:00

facebook_chat.py

community[minor]: Implement lazy_load() for FacebookChatLoader (#18669)

2024-03-06 09:15:00 -05:00

fauna.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

figma.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

gcs_directory.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

gcs_file.py

fix: correct spelling mistakes of "seperate, intialise, pre-defined" (#14647)

2023-12-22 11:49:35 -08:00

generic.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

geodataframe.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

git.py

Merge pull request #18539

2024-03-06 13:25:14 -05:00

gitbook.py

community[minor]: Implement lazy_load() for GitbookLoader (#18670)

2024-03-06 09:14:36 -05:00

github.py

community: Implement lazy_load() for GithubFileLoader (#18584)

2024-03-05 09:35:50 -08:00

google_speech_to_text.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

googledrive.py

infra: add print rule to ruff (#16221)

2024-02-09 16:13:30 -08:00

gutenberg.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

helpers.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

hn.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

html_bs.py

Merge pull request #18423

2024-03-06 13:25:01 -05:00

html.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

hugging_face_dataset.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

hugging_face_model.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

ifixit.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

image_captions.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

image.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

imsdb.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

iugu.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

joplin.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

json_loader.py

community: Implement lazy_load() for JSONLoader (#18643)

2024-03-08 13:58:17 -05:00

lakefs.py

docs: docstrings langchain_community update (#14889)

2023-12-19 08:58:24 -05:00

larksuite.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

markdown.py

corrected outdated link (#15053)

2023-12-22 12:39:38 -08:00

mastodon.py

Merge pull request #18671

2024-03-06 13:23:14 -05:00

max_compute.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

mediawikidump.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

merge.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

mhtml.py

community[patch]: Implement lazy_load() for MHTMLLoader (#18648)

2024-03-07 11:50:18 -05:00

modern_treasury.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

mongodb.py

community[minor]: added a feature to filter documents in Mongoloader (#18253)

2024-03-08 12:06:35 -08:00

news.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

notebook.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

notion.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

notiondb.py

community[patch]: support query filters for NotionDBLoader (#17217)

2024-02-14 11:43:41 -08:00

nuclia.py

infra: add print rule to ruff (#16221)

2024-02-09 16:13:30 -08:00

obs_directory.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

obs_file.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

obsidian.py

Merge pull request #18654

2024-03-06 13:06:55 -05:00

odt.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

onedrive_file.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

onedrive.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

onenote.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

open_city_data.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

org_mode.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

pdf.py

Merge pull request #18673

2024-03-06 13:24:36 -05:00

pebblo.py

community[patch]: Fix pwd import that is not available on windows (#17532)

2024-02-14 13:45:10 -08:00

polars_dataframe.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

powerpoint.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

psychic.py

Merge pull request #18656

2024-03-06 13:05:04 -05:00

pubmed.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

pyspark_dataframe.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

python.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

quip.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

readthedocs.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

recursive_url_loader.py

community[patch]: fix RecursiveUrlLoader metadata_extractor return type (#18193)

2024-03-01 12:08:20 -08:00

reddit.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

roam.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

rocksetdb.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

rspace.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

rss.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

rst.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

rtf.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

s3_directory.py

community[patch]: Skip nested directories when using S3DirectoryLoader (#17829)

2024-03-08 16:50:58 -08:00

s3_file.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

sharepoint.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

sitemap.py

community[minor]: Implement lazy_load() for SitemapLoader (#18667)

2024-03-06 09:15:35 -05:00

slack_directory.py

community: Implement lazy_load() for SlackDirectoryLoader (#18675)

2024-03-06 13:04:13 -05:00

snowflake_loader.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

spreedly.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

sql_database.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

srt.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

stripe.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

surrealdb.py

community[patch]: SurrealDB fix for asyncio (#16092)

2024-01-23 19:46:19 -08:00

telegram.py

text-splitters[minor], langchain[minor], community[patch], templates, docs: langchain-text-splitters 0.0.1 (#18346)

2024-02-29 18:33:21 -08:00

tencent_cos_directory.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

tencent_cos_file.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

tensorflow_datasets.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

text.py

Merge pull request #18674

2024-03-06 13:23:42 -05:00

tidb.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

tomarkdown.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

toml.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

trello.py

community: Implement lazy_load() for TrelloLoader (#18658)

2024-03-06 13:04:36 -05:00

tsv.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

twitter.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

unstructured.py

Merge pull request #18647

2024-03-06 13:13:10 -05:00

url_playwright.py

community: Implement lazy_load() for PlaywrightURLLoader (#18676)

2024-03-06 16:52:13 -05:00

url_selenium.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

url.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

vsdx.py

community[minor]: New documents loader for visio files (with extension .vsdx) (#16171)

2024-01-22 22:07:03 -08:00

weather.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

web_base.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

whatsapp_chat.py

community: Implement lazy_load() for WhatsAppChatLoader (#18677)

2024-03-06 13:03:46 -05:00

wikipedia.py

community[minor]: Implement lazy_load() for WikipediaLoader (#18680)

2024-03-06 13:03:21 -05:00

word_document.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

xml.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

xorbits.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

youtube.py

community[patch]: docstrings (#16810)

2024-02-09 12:48:57 -08:00

yuque.py

community[minor]: add Yuque document loader (#17924)

2024-03-05 15:54:07 -08:00