[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/treebanks/ja/bccwjluw/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [treebanks](https://md.mova.elamur.ai/free/en/docs/treebanks/) / [ja](https://md.mova.elamur.ai/free/en/docs/treebanks/ja/) / UD_Japanese-BCCWJLUW

- [papers.md](https://md.mova.elamur.ai/free/en/docs/treebanks/ja/bccwjluw/papers/) — Papers: UD_Japanese-BCCWJLUW

# UD_Japanese-BCCWJLUW

Treebank card from UD release 2.18: metadata and texts come from `UD_Japanese-BCCWJLUW/README.md`; size is counted by streaming the `.conllu` files and checked against the release `stats.xml`. Generated by `tools/treebanks build`; do not edit by hand. Papers: [Papers: UD_Japanese-BCCWJLUW](https://md.mova.elamur.ai/free/en/docs/treebanks/ja/bccwjluw/papers/); seeds: `seeds/`; converter: `convert.md`.

| Field | Value |
| --- | --- |
| UD name | UD_Japanese-BCCWJLUW |
| Language | Japanese (`ja`) |
| Family | Japanese |
| Treebank code | `ja_bccwjluw` |
| UD page | [universaldependencies.org/treebanks/ja bccwjluw/index.html](https://universaldependencies.org/treebanks/ja_bccwjluw/index.html) |
| License | CC BY-NC-SA 4.0 |
| Genres | news, nonfiction, fiction, blog, web |
| Text in data | no |
| Parallel | no |
| In UD since | UD v2.9 |
| Authors | Mai Omura; Masayuki Asahara; Yusuke Miyao; Takaaki Tanaka; Hiroshi Kanayama; Yuji Matsumoto; Shinsuke Mori; Sumire Uematsu; Yugo Murawaki |
| Contact | [masayu-a@ninjal.ac.jp](mailto:masayu-a@ninjal.ac.jp) |
| Development | outside the UD repo (elsewhere): fixes go to the source or the converter |

## Annotation layers

| Layer | Origin |
| --- | --- |
| Lemmas | not available |
| UPOS | converted from manual |
| XPOS | not available |
| Features | not available |
| Relations | converted from manual |

UD scale: *manual native* — annotated by hand directly in UD; *converted from manual* — converted from manual annotation in another scheme; *converted with corrections* — converted and partly corrected by hand; *automatic with corrections* — automatic, with partial manual corrections; *automatic* — automatic; *not available* — the layer is absent.

## Size

| Split | Sentences | Tokens | Words | Multiword tokens | Empty nodes |
| --- | --- | --- | --- | --- | --- |
| train | 40 801 | 715 759 | 715 759 | 0 | 0 |
| dev | 8 427 | 145 398 | 145 398 | 0 | 0 |
| test | 7 881 | 134 475 | 134 475 | 0 | 0 |
| **total** | 57 109 | 995 632 | 995 632 | 0 | 0 |

Tokens are surface tokens (Spanish `vámonos` is one), words are syntactic words (`vamos` + `nos`); multiword tokens are those split into several words; empty nodes are the `N.M` nodes of the enhanced graph.

## Summary

> This Universal Dependencies (UD) Japanese treebank is based on the definition of UD Japanese convention described in the UD documentation. The original sentences are from `Balanced Corpus of Contemporary Written Japanese'(BCCWJ).
> 
> UD-Japanese-BCCWJLUW is the other word segmentation version of UD-Japanese-BCCWJ. We use **Long Unit Word (LUW)** as their syntactic word in UD definition.

## Introduction

> The Japanese UD treebank contains the sentences from BCCWJ [1] [clrd.ninjal.ac.jp/bccwj/en](https://clrd.ninjal.ac.jp/bccwj/en/) with BCCWJ-DepPara[2] annotation.
> 
> The word units is based on **Long Unit Word** in BCCWJ [1]. We prepared conversion rules from BCCWJ-DepPara to UD_Japanese v2.1 guidelines [3][4].
> 
> ## Recovering data
> 
> The data is provided in the CoNLL format, but original texts are stripped off due to the license issue.
> 
> The corpus is obtained by running the following command:
> 
> ```
> ./merge/merge.sh -c BCCWJ_CORE_FILE
> ```
> 
> where `BCCWJ_CORE_FILE` denotes the file of BCCWJ core file (core_LUW.txt, OT) the script requires Python (3.x)
> 
> or
> 
> The BCCWJ DVD edition purchaser can download the data with the original text from [bccwj-data.ninjal.ac.jp](https://bccwj-data.ninjal.ac.jp/)
> 
> ## Spliting
> 
> Each data set contains UD annotations for the following parts in BCCWJ
> 
> training: annotation B, C, D, E subsets development: annotation A subset test: annotation A subset
> 
> See also [github.com/masayu-a/BCCWJ-ANNOTATION-ORDER](https://github.com/masayu-a/BCCWJ-ANNOTATION-ORDER)

- [← previous UD_Japanese-BCCWJ](https://md.mova.elamur.ai/free/en/docs/treebanks/ja/bccwj/)
- [next → UD_Japanese-GSD](https://md.mova.elamur.ai/free/en/docs/treebanks/ja/gsd/)
- [↑ back to Japanese (ja) — UD 2.18 treebanks](https://md.mova.elamur.ai/free/en/docs/treebanks/ja/)