Hub documentation

๐Ÿค— Datasets ์‚ฌ์šฉ

Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

๐Ÿค— Datasets ์‚ฌ์šฉ

Hugging Face Hub์—์„œ ํฅ๋ฏธ๋กœ์šด ๋ฐ์ดํ„ฐ์…‹์„ ์ฐพ์•˜๋‹ค๋ฉด, ๐Ÿค— Datasets๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ํ•ด๋‹น ๋ฐ์ดํ„ฐ์…‹์„ ๋กœ๋“œํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์ด ๋ฐ์ดํ„ฐ์…‹ ์‚ฌ์šฉ ๋ฒ„ํŠผ์„ ํด๋ฆญํ•˜๋ฉด ๋ฐ์ดํ„ฐ์…‹์„ ๋กœ๋“œํ•˜๋Š” ์ฝ”๋“œ๊ฐ€ ๋ณต์‚ฌ๋ฉ๋‹ˆ๋‹ค.

๋จผ์ € Hugging Face ๊ณ„์ •์œผ๋กœ ๋กœ๊ทธ์ธํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. ์˜ˆ๋ฅผ ๋“ค์–ด ๋‹ค์Œ์„ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค:

hf auth login

๊ทธ ํ›„ Hugging Face Hub์—์„œ load_dataset๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋ฐ์ดํ„ฐ์…‹์„ ๋กœ๋“œํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค

from datasets import load_dataset

dataset = load_dataset("username/my_dataset")

# or load the separate splits if the dataset has train/validation/test splits
train_dataset = load_dataset("username/my_dataset", split="train")
valid_dataset = load_dataset("username/my_dataset", split="validation")
test_dataset  = load_dataset("username/my_dataset", split="test")

Hugging Face Hub์— ๋ฐ์ดํ„ฐ์…‹์„ ์—…๋กœ๋“œํ•  ์ˆ˜๋„ ์žˆ์Šต๋‹ˆ๋‹ค:

my_new_dataset.push_to_hub("username/my_new_dataset")

์ด ์ž‘์—…์€ ๋‚˜์ค‘์— ๋‹ค์‹œ ๋กœ๋“œํ•  ์ˆ˜ ์žˆ๋Š” Parquet ํ˜•์‹์˜ Dataset์„ ํฌํ•จํ•˜๋Š” ๋ฐ์ดํ„ฐ์…‹ ์ €์žฅ์†Œ username/my_new_dataset๋ฅผ ์ƒ์„ฑํ•ฉ๋‹ˆ๋‹ค.

๐Ÿค— Datasets ์‚ฌ์šฉ์— ๋Œ€ํ•œ ์ž์„ธํ•œ ๋‚ด์šฉ์€ ๐Ÿค— Datasets ๋ฌธ์„œ์—์„œ ์ œ๊ณตํ•˜๋Š” ํŠœํ† ๋ฆฌ์–ผ๊ณผ ๊ฐ€์ด๋“œ๋ฅผ ์ฐธ๊ณ ํ•˜์„ธ์š”.

Update on GitHub