Transformer-XL

Google/CMU · 2019 · LLM · USA

A to Z

A · App: No
B · Built By: Google/CMU
C · CEO & Politics: [UNKNOWN]
D · Date: 2019
E · Expertise: Intermediate+
F · Free Tier: Yes
G · Generates: Text only
H · Deployment: Cloud + Local (open weights)
I · Inputs: Text
J · Jurisdiction: USA
K · Key Staffers: Zihang Dai (Co-lead author), Zhilin Yang (Co-lead author), Quoc V. Le (Senior author), Ruslan Salakhutdinov (Senior author)
L · Languages: English-primary
M · Model Type: LLM
N · Safety Issues: None reported
O · Open Source: Yes — open weights
P · Pricing: Free (open source, Apache 2.0 license)
Q · AIDb Rating: [UNKNOWN]
R · What Makes It Special: Transformer-XL, a language-model architecture that adds segment-level recurrence and a new positional encoding scheme so Transformers can learn dependencies beyond a fixed-length context; it learns dependencies 80% longer than RNNs and 450% longer than vanilla Transformers, evaluates up to 1,800+ times faster, and set state-of-the-art results on enwik8, text8, WikiText-103, One Billion Word and Penn Treebank.
S · Stock / Status: Research / open-source project
T · Training Cutoff: [UNKNOWN]
U · Users: [UNKNOWN]
V · Voice Chat: No
W · Weights: Downloadable
X · X Handle: [UNKNOWN]
Y · Year Founded: [UNKNOWN]
Z · Bottom Line: Open-weights LLM from Google/CMU, released 2019.

Data completeness 77% · Last verified 2026-05-12 · Primary source ↗

“Transformer-XL.” AIDb — The AI Database. Verified 2026-05-12. https://theaidatabase.com/m/transformer-xl.html