[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/treebanks/ja/gsd/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [treebanks](https://md.mova.elamur.ai/free/en/docs/treebanks/) / [ja](https://md.mova.elamur.ai/free/en/docs/treebanks/ja/) / UD_Japanese-GSD

- [papers.md](https://md.mova.elamur.ai/free/en/docs/treebanks/ja/gsd/papers/) — Papers: UD_Japanese-GSD

# UD_Japanese-GSD

Treebank card from UD release 2.18: metadata and texts come from `UD_Japanese-GSD/README.md`; size is counted by streaming the `.conllu` files and checked against the release `stats.xml`. Generated by `tools/treebanks build`; do not edit by hand. Papers: [Papers: UD_Japanese-GSD](https://md.mova.elamur.ai/free/en/docs/treebanks/ja/gsd/papers/); seeds: `seeds/`; converter: `convert.md`.

| Field | Value |
| --- | --- |
| UD name | UD_Japanese-GSD |
| Language | Japanese (`ja`) |
| Family | Japanese |
| Treebank code | `ja_gsd` |
| UD page | [universaldependencies.org/treebanks/ja gsd/index.html](https://universaldependencies.org/treebanks/ja_gsd/index.html) |
| License | CC BY-SA 4.0 |
| Genres | news, blog |
| Text in data | yes |
| Parallel | jagsd |
| In UD since | UD v1.4 |
| Authors | Mai Omura; Yusuke Miyao; Hiroshi Kanayama; Hiroshi Matsuda; Aya Wakasa; Kayo Yamashita; Masayuki Asahara; Takaaki Tanaka; Yugo Murawaki; Yuji Matsumoto; Shinsuke Mori; Sumire Uematsu; Ryan McDonald; Joakim Nivre; Daniel Zeman |
| Contact | [hkana@jp.ibm.com](mailto:hkana@jp.ibm.com) |
| Development | in the UD repo (here) |

## Annotation layers

| Layer | Origin |
| --- | --- |
| Lemmas | converted from manual |
| UPOS | converted from manual |
| XPOS | manual native |
| Features | not available |
| Relations | converted from manual |

UD scale: *manual native* — annotated by hand directly in UD; *converted from manual* — converted from manual annotation in another scheme; *converted with corrections* — converted and partly corrected by hand; *automatic with corrections* — automatic, with partial manual corrections; *automatic* — automatic; *not available* — the layer is absent.

## Size

| Split | Sentences | Tokens | Words | Multiword tokens | Empty nodes |
| --- | --- | --- | --- | --- | --- |
| train | 7 050 | 168 333 | 168 333 | 0 | 0 |
| dev | 507 | 12 287 | 12 287 | 0 | 0 |
| test | 543 | 13 034 | 13 034 | 0 | 0 |
| **total** | 8 100 | 193 654 | 193 654 | 0 | 0 |

Tokens are surface tokens (Spanish `vámonos` is one), words are syntactic words (`vamos` + `nos`); multiword tokens are those split into several words; empty nodes are the `N.M` nodes of the enhanced graph.

## Summary

> This Universal Dependencies (UD) Japanese treebank is based on the definition of UD Japanese convention described in the UD documentation. The original sentences are from Google UDT 2.0.

## Introduction

> The Japanese UD treebank contains the sentences from Google Universal Dependency Treebanks v2.0 (legacy): [github.com/ryanmcd/uni-dep-tb](https://github.com/ryanmcd/uni-dep-tb). First, Google UDT v2.0 was converted to UD-style with bunsetsu-based word units (say "master" corpus).
> 
> The word units in "master" is significantly different from the definition of the documents based on **Short Unit Word** (SUW) [1], then the sentences are automatically re-processed by Hiroshi Kanayama in Feb 2017. It is the Japanese_UD v2.0 and used in the CoNLL 2017 shared task. In November 2017, UD_Japanese v2.0 is merged with the "master" data so that the manual annotations for dependencies can be reflected to the corpus. It reduced the errors in the dependency structures and relation labels.
> 
> Still there are slight differences in the word unit between UD_Japanese v2.1 and UD_Japanese-KTC 1.3.
> 
> In May 2020, we introduce UD_Japanese BCCWJ[3] like coversion method for UD_Japanese GSD v2.6.

- [← previous UD_Japanese-BCCWJLUW](https://md.mova.elamur.ai/free/en/docs/treebanks/ja/bccwjluw/)
- [next → UD_Japanese-GSDLUW](https://md.mova.elamur.ai/free/en/docs/treebanks/ja/gsdluw/)
- [↑ back to Japanese (ja) — UD 2.18 treebanks](https://md.mova.elamur.ai/free/en/docs/treebanks/ja/)