Common Crawl Foundation @CommonCrawl
Common Crawl is a non-profit foundation dedicated to the Open Web. commoncrawl.org San Francisco, CA Joined February 2010-
Tweets2K
-
Followers8K
-
Following2K
-
Likes705
0
0
1
273
1
View Details
www2.statmt.org/wmt26/pdf/2026… We present the results of the WMT 2026 shared task of the Open Language Data Initiative (OLDI). Participants were invited to contribute to the massively multilingual open datasets supported by OLDI, FLORES+ and OLDI Seed, or to create new resources in line with OLDI’s ethos. We accepted ten submissions: five extending FLORES+, and five extending other open and massively-parallel datasets (BOUQuET, Seed, SMOL, and WMT24+). These contributions advance the coverage and quality of multilingual datasets, especially for underserved language varieties. All contributions are released under permissive open-source licenses.
Findings of the WMT 2026 Shared Task of the Open Language Data Initiative Laurie Burchell Common Crawl Foundation
Common Crawl Foundation retweeted
@CommonCrawl My bet: low-resource languages would lose at the filter. Quality classifiers trained mostly on English throw out good Icelandic pages as noise. Smarter crawl budgets won't help if the filter can't read the language.
The Common Crawl Foundation at the TPDL Conference The Common Crawl Foundation participated in the 30th TPDL (Theory and Practice for Digital Archives) conference in Faro, Portugal presenting two contributions on Crawl theory and on archival data enrichment.
arxiv.org/abs/2610.13321 Web crawling is a major source of large language model (LLM) pretraining data, yet data-selection pipelines discard a large share of crawled pages. We investigate how a crawler should allocate its budget across and within hosts to collect more high-quality pretraining data.
Common Crawl - the ultimate time capsule for knowledge | Amazon Web Services youtube.com/watch?v=rGT-al…
Common Crawl Foundation retweeted
we will have two papers @NeurIPSConf this year from the croissant community 🥐 1. "croissant baker" led by rafi al attrach and @tompollard 2. "croissant miner" led by berke arda and @akhtarmubashara i took a quick glance at the rearview mirror since early 2023, our idea for a unified ml data access layer has grown into an expansive ecosystem, incl - data aggregators spanning hundreds of thousands of datasets incl @huggingface @kaggle @dataverseorg @CommonCrawl and others - tooling spanning auto-generation and -verification of croissant files - the formal spec maintenance under @MLCommons - and a growing list of community extensions spanning verticals such geo, responsible AI, health and others onwards towards the composable data dream(:
Common Crawl Foundation retweeted
There’s an enormous amount of open training data on Hugging Face. What does it take to make it work together 🤗? For Marin’s 535B run, we built on 25T tokens from 152 datasets with licenses permitting training. Here’s the work between downloading those and training a model 🧵
Information Stewardship Forum 2026: Creating Community and Purpose Around US Government Information A report from the inaugural Information Stewardship Forum, a three-day convening hosted by the Internet Archive in San Francisco on March 18–20, 2026, that brought together 120 people working to preserve and provide access to United States government information.
September 2026 Crawl Archive Now Available We are pleased to announce the release of the September 202 6 crawl archive, consisting of 2.17 billion pages (or 361.4 TiB of uncompressed content.) 📷
Getting Started with Common Crawl Data on Hugging Face We made parts of our crawl archive available experimentally on a Hugging Face Storage Bucket. This blog post explains how to use the data on Hugging Face, with tools and examples on which you can build.
CommonsDB, ISCC, Spaghetti, and Meatballs Notes from the CommonsDB final conference in Alicante: a registry of 6.5 million open works keyed on ISCC, and the EUIPO's push for federated copyright infrastructure.
Jeremy Howard @jeremyphoward
331K Followers 7K Following 🇦🇺 Co-founder: @AnswerDotAI/@FastDotAI ; Prev: Professor@UQ; @kaggle founding president; founder @fastmail/@enlitic/… https://t.co/16UBFTX7mo
Antonio García Mart�... @antoniogm
230K Followers 17K Following Director, @base growth. Founder @spindl_xyz (acq. @coinbase). Wrote bestseller 'Chaos Monkeys'. גם זה יעבור 🇺🇸🇪🇸
Michael Nielsen @michael_nielsen
123K Followers 5K Following Searching for the numinous 🇦🇺 🇨🇦, currently live in 🇺🇸 Research @AsteraInstitute https://t.co/maezekzk4D
Michael L. Nelson @phonedude_mln
2K Followers 953 Following Professor: @WebSciDL @ODUcs @ODUVMASC @ODUDataScience (2002-now); Engineer: @NASA_Langley (1991-2002); Postdoc: @UNCSILS (2000-2001)
Peter Wang 🦋 @pwang
48K Followers 2K Following Chief AI & Co-founder @AnacondaInc; invented @pyscript_dev, @PyData @Bokeh @Datashader. Former physicist. A student of the human condition. bsky: @wang.social
👩💻 Paige Bai... @DynamicWebPaige
76K Followers 2K Following ✨AI should be about empowering humans, building understanding, and making dreams realities. 👩💻 DevRel Eng. Lead @GoogleDeepMind ex-@GitHub || views = my own!
𖦹p卐imys卍iΦ @0xptimystic
531 Followers 5K Following 🏳️🏳️🏳️RETARDMAXXING 🏳️🏳️🏳️ 🏳️🏳️SURRENDERMAXXING🏳️🏳️ 🏳️🏳️🏳️SCHIZOMAXXING 🏳️🏳️🏳️ 🏳degen(pba)/(regen/refi/desci)🏳️
AgentWebMap @AgentWebMap
1 Followers 16 Following
Project Timestamper @projtimestamper
69 Followers 43 Following Assuring the integrity of humanity's cultural and scientific heritage.
615525352671 @615525352671Q
8 Followers 2K Following
slm tokens @tulkenss
518 Followers 832 Following Chief whatever officer @minishlab Smallest model manufacturer Silly clown Citizen of Antwerp
Al Emran @ALEMRANCU
198 Followers 674 Following CTO at ReachMark | Software Architect | Helping startups plan & build clean, cost-effective SaaS
SPN @spnichol
847 Followers 6K Following Machine learning engineer working on signal processing, NLP and (some) computer vision. Trying to make the most of my measley 3090 and 4 Sparks.
Nike Long @nikelong
187 Followers 6K Following Analytics. Cryptomarket research. Made CryptoAlcoholics & @Bl0ckchainUA conference with heart.
chichi wang @chichiwang81259
0 Followers 32 Following
riaan kumar @RiaanKumar
65 Followers 313 Following building in data @comprehensionAI | studied economics of data @ucberkeley
Christian Hoang @christhoang04
94 Followers 1K Following My calling is to contribute as much as I can, to humanity.
kyzance @kyzance
47 Followers 1K Following
Luke Tofts @LukeTofts
75 Followers 330 Following
ProbablePrime @ProbablePrime
1K Followers 1K Following Co-Founder @ResoniteApp Engineer. Previously @WatchMixer, @TrackIntel. VR, XR, Metaverse. My opinions are mine and about cheese. Something new at: https://t.co/zqS0XXRmtK
hgtp://chriswiig @ChristofferWii4
250 Followers 735 Following $DAG $ADS $UNI $LINK $ADA $POL $PEPE $AVAX
ISOforthewin @IOforthewi42160
1K Followers 1K Following CHRIST IS KING, $XRP vibes, $DAG, Psalms 37:7, 🪙
WHTTW @WHTTWamg
15 Followers 280 Following
Sophia @Ahmedmo88180965
2 Followers 712 Following true crime podcasts, oat milk lattes & the occasional bad idea ☕ mostly harmless
goubie @goutoberry
145 Followers 843 Following building agentic systems, and also co-founder and cto at b2g startup.
Rishi @Rishi_S1
0 Followers 478 Following
Posnofa @Afonsop4
102 Followers 2K Following
WNTR.ai @wntrai
764 Followers 4K Following AI jobs at the click of a prompt. Fresh roles from companies building AI, updated every day.
David Hall @dlwh
4K Followers 1K Following Creator of Levanter and @MarinCommunity. MTS @ Open Athena. Previously Research Engineering @StanfordCRFM, co-founder at Semantic Machines ⟶ MSFT.
Jerry Blount @jerryblount
76 Followers 1K Following "We can't solve problems by using the same kind of thinking we used when we created them." - Albert Einstein
BECOOL @becool_me
2K Followers 825 Following https://t.co/ekfw9Z86kx https://t.co/CEWH7kkqit https://t.co/4MGqYU4Ne4 https://t.co/9eAv3MZBXn https://t.co/Sr2fagnh5H https://t.co/VGQbhr1gte https://t.co/w1QCsTDZYR 英文号：@we_webmaster
Will Held @WilliamBarrHeld
3K Followers 1K Following Open LLM Training @ https://t.co/yb9OySgHFM Formerly ML PhD w/ @Diyi_Yang, 🦙 @AIatMeta, Assistant @GoogleAI, اللغة العربية @NYUAbuDhabi Burqueño
Monty @montyjinspired
88 Followers 5K Following
Leonard @lionheartlurve
11 Followers 2K Following
Bajoran worker @portalsfra
67 Followers 3K Following
supperai @supperai130317
15 Followers 1K Following
janrk 🦣 @janrk92
149 Followers 3K Following
RAZA | AI EXPLORER @razaaitech
65K Followers 13K Following AI Influencer | SaaS Tools Reviews & Demos Testing AI tools, creating videos & sharing honest insights DM for paid collab 📧 [email protected]
BeLeap @bel3ap
45 Followers 214 Following Creating something unnecessary Mastodon: https://t.co/pmWlSkEVSW Bluesky: https://t.co/FCuTSnb4t2 GitHub: https://t.co/0Ss7ySx7r5
Fadil Khodabuccus @fkhoda
103 Followers 2K Following Principal Engineer @Checkout. Opinions are my own.
James Thornton @itsjthornton
11 Followers 68 Following CEO, Daz 3D + Tafi. Founder, The AI Ranch. Building licensed, rights-clean training data and proprietary tech for AI. Host, Cowboy AI (coming soon)
Rahul Atlury @atlury
87 Followers 3K Following I am an electronics engineer with interests in preserving the knowledge of yesteryears for future generations
Sam Altman @sama
6.4M Followers 1K Following The mission of OpenAI is to ensure that AGI benefits all of humanity
François Chollet @fchollet
749K Followers 831 Following Co-founder @ndea. Co-founder @arcprize. Creator of Keras and ARC-AGI. Author of 'Deep Learning with Python'.
Andrej Karpathy @karpathy
4.4M Followers 1K Following
Yann LeCun @ylecun
1.3M Followers 800 Following Founder/Chair, AMI Labs; Professor, NYU; Partner, 224 Ventures; Ex-Chief AI Scientist, Meta. Researcher in AI, ML, Robotics, etc. ACM Turing Award Laureate.
Bojan Tunguz @tunguz
306K Followers 9K Following Founder and CEO @tabul_ai. Creator of @trainxgb. ML ex Nvidia. Data Scientist. Physicist. Catholic. Husband. Father. Stanford Alum. Memelord. e/xgb. AMDG.
Andrew Ng @AndrewYNg
1.9M Followers 1K Following Co-Founder of Coursera; Stanford CS adjunct faculty. Former head of Baidu AI Group/Google Brain. #ai #machinelearning, #deeplearning #MOOCs
Jason Scott @textfiles
52K Followers 629 Following Proprietor of https://t.co/sdyjXHCZF7, historian, filmmaker, archivist, storyteller. Works on/for the Internet Archive. Rank Amateur. Pitiful Man.
Jeff Atwood @codinghorror
262K Followers 2 Following Indoor enthusiast. Co-founder https://t.co/e62S5uByfO / https://t.co/Tuh5wHPHTI. Let’s be kind to each other. I am no longer on twitter. Find me @[email protected]
GitHub @github
2.7M Followers 334 Following The AI-powered developer platform to build, scale, and deliver secure software.
Jeremy Howard @jeremyphoward
331K Followers 7K Following 🇦🇺 Co-founder: @AnswerDotAI/@FastDotAI ; Prev: Professor@UQ; @kaggle founding president; founder @fastmail/@enlitic/… https://t.co/16UBFTX7mo
Percy Liang @percyliang
127K Followers 424 Following professor of computer science @Stanford @stanfordnlp, co-founder of @togethercompute, https://t.co/7R5THVnJ6u, @simile_ai, pianist
roon @tszzl
507K Followers 14K Following “ceterum censeo we must pace the frontier of global machine intelligence progress”
Jeff Dean @JeffDean
531K Followers 6K Following Co-founder & CEO of Discovery Loop. Former Chief Scientist, Google. Helped build many Google products, TPUs, Gemini, TensorFlow, MapReduce, Bigtable, ...
Internet Archive @internetarchive
506K Followers 1K Following Internet Archive is a non-profit research library preserving web pages, books, movies & audio for public access. Explore web history via the @waybackmachine.
Michael Nielsen @michael_nielsen
123K Followers 5K Following Searching for the numinous 🇦🇺 🇨🇦, currently live in 🇺🇸 Research @AsteraInstitute https://t.co/maezekzk4D
Michael L. Nelson @phonedude_mln
2K Followers 953 Following Professor: @WebSciDL @ODUcs @ODUVMASC @ODUDataScience (2002-now); Engineer: @NASA_Langley (1991-2002); Postdoc: @UNCSILS (2000-2001)
Peter Wang 🦋 @pwang
48K Followers 2K Following Chief AI & Co-founder @AnacondaInc; invented @pyscript_dev, @PyData @Bokeh @Datashader. Former physicist. A student of the human condition. bsky: @wang.social
Jimmy Lin @lintool
15K Followers 876 Following I profess CS-ly at @UWaterloo. Previously, I monkeyed code for @Twitter, slides for @Cloudera, and scienced for @yupp_ai.
Brian Wesley @BrianWesley0711
222 Followers 2K Following NYC uni · AI/NLP + linguistics · learning in public · EN/中文 · building toward something useful
Will Held @WilliamBarrHeld
3K Followers 1K Following Open LLM Training @ https://t.co/yb9OySgHFM Formerly ML PhD w/ @Diyi_Yang, 🦙 @AIatMeta, Assistant @GoogleAI, اللغة العربية @NYUAbuDhabi Burqueño
Ryan Jones @RyanJones
13K Followers 1K Following SVP, SEO & Stable Genius @ Razorfish. Founder https://t.co/hiC5n0dzWE Works from early in the morning until late in the evening. Makes many calls and has many meetings.
swyx @swyx
204K Followers 4K Following achieve ambition with intentionality, intensity, integrity & insanity. affiliations: - @smol_ai - @dxtipshq - @cognition - @aidotengineer - @latentspacepod
Gabriel Altay @gabrielaltay
404 Followers 2K Following Multimodal ML + Bio @TempusAI | Computational Physics PhD from @CarnegieMellon
Siddharth @siddharthvader_
1K Followers 2K Following partner @scalevp | prev: ml engineer, math & cs @columbia | I ♥️ SF & bay area sports
fils @fils
359 Followers 1K Following San Diego Supercomputer Center (UCSD/SDSC) Research Data Services / Data Initiatives Group
suchit @asquirous
75 Followers 903 Following curiosity killed the cat — but the cat had 9 lives. learns ML and builds stuff.
Pew Research Center @pewresearch
417K Followers 208 Following Nonpartisan, nonadvocacy data and analysis. Follow us on Instagram, Facebook and LinkedIn or subscribe to our newsletters: https://t.co/tawtYBr8Ef ✉️
FÄRR @farrclick
266 Followers 420 Following indie generalist. working across interfaces, computation, and whatever rabbit holes comes next.
Jalal Ashrafi @jalalash
15K Followers 112 Following Senior Atlassian engineer. 20+ years in Java, Python and startups. Building agent products; exploring context, memory and provenance. Views mine.
Kabo Kable Molefe @kabokablemolefe
626 Followers 883 Following Proponderance. Level 10 Operator. Remember the name. Deal Maker. Rainmaker
Àrẹ̀mú Adéọl�... @aremuadeolajr
1K Followers 1K Following Computational Linguist | Resp AI Fellow @StimsonCenter | NLP Researcher @MasakhaneNLP | MSc @cimec_unitrento | Curating large-scale dataset for African Lang. 🌍
Vukosi Marivate 🇿�... @vukosi
14K Followers 1K Following Professor NLP/DS/AI @UPTuks, Co-Founder @LelapaAI. PhDCS Rutgers MsEE Wits. Changing the World @DeepIndaba @MasakhaneNLP ♥️️Tshwane Lab @dsfsi_research ☠️
IDRC | CRDI @IDRC_CRDI
44K Followers 1K Following International Development Research Centre / Le Centre de recherches pour le développement international.
arXiv.org @arxiv
56K Followers 184 Following News from https://t.co/enurGFxpcS, a free distribution service and an open archive for scholarly articles. For help with arXiv, see https://t.co/LcWuhM0BOl
Abe Voelker @abevoelker
717 Followers 522 Following Programming, ISP/telecom, rural life stuff. Ex farm kid
arockoholic² @arockoholic
2K Followers 6K Following Dad, Gamer, Metalhead, Humor—Opinions are my own. Trust in Data is Key - Our children deserve an honest future. 🌌 #hgtp 🤘 $DAG 👾 $INT
Ray Martinez @RayMartinezSEO
897 Followers 713 Following 🇵🇷 - SEO. Music Producer. DJ. Traveler. Personal Twitter. VP of SEO - @archereducation | Adjunct Lecturer at UF | Author
YounesIO @YounesAka
140 Followers 703 Following SWE, AI agents & formal methods. Software is about semantics. On macOS & Omarchy.
Hamdy Mubarak @Hamdy_Mubarak
849 Followers 2K Following Principal Software Engineer at Qatar Computing Research Institute
ujj bsky social @uinelj
161 Followers 629 Following data @poolsideai find me at butterfly site: https://t.co/wYEOzbuhaQ
Daniel van Strien @vanstriendaniel
7K Followers 1K Following Machine Learning Librarian @huggingface 🤗 I like big datasets and small models.
Mike Hutu @MichaelHutu
301 Followers 536 Following AI Native Dev | Open-source hunter | Focused on building with- and on top of foundation & local models | Keeping up with the AI era | 2x DGX Sparks
Tristan Rhodes @tristanbob
4K Followers 7K Following AI engineer, senior vibe coder, open source advocate, network engineer, game dev, trumpeter, pickleball, and dad. https://t.co/kq2xKmtbNN
Bê Acosta👩💻 @britneyscripts
1K Followers 580 Following Product Manager 🇧🇷🇺🇾| 8+ YOE |MBA Data Science @icmc_usp | UTFPR & ESPM Alumni | Researching Agentic Commerce ✍🏼 https://t.co/DBlZs8mqVU
Andrew Piper @_akpiper
6K Followers 3K Following Using #AI and #NLP to study storytelling at McGillU. Director of .txtlab and author of the new book, Why You Should Read More Fiction.
Judd Burton @burtonbeyond
5K Followers 132 Following Dr. Judd H. Burton is an historian, anthropologist, & archaeologist. His research covers a range of supernatural & conventional topics within the humanities.
Lutfi Zuchri @lutfizuchri
1K Followers 984 Following Nose-picking enthusiast. JRPG aficionado. E-commerce, Digital Strategy, and accidental Data Science guy. ex @TripAdvisor @Teespring
Ariane Bibel 🍵 @ArianeKonzepte
824 Followers 885 Following Kommunikations- & Medienwissenschaftlerin 👋 | Angeh. Konzepterin | Strategien für: deutsche Technik, Ingenieur, Forschung & KMU | Kurator @medienfeed
Joe Devon @joedevon
6K Followers 6K Following #GAAD (Global Accessibility Awareness Day) 200M+ social media reach AIMAC, an AI benchmark for A11y @A11yGenAI my podcast @a11yaudits
EleutherAI @AiEleuther
29K Followers 107 Following
Matt Perault @MattPerault
3K Followers 642 Following Head of AI Policy @a16z. Fmr director of @UNC_TechPolicy and Facebook public policy.
Snibby @ItsSnibby
27K Followers 14K Following research → conviction → chaos portfolio powered by pure denial 🦅
Tommaso Green @tommasogreen
193 Followers 1K Following PhD Student 👨🎓 in NLP 📜 at the University of Mannheim.
Joseph Imperial @josephimperial_
2K Followers 6K Following AI safety, governance, and law. Visiting Researcher @law_ai_ ⚖️ PhD Candidate @ARTAIBath 🇬🇧 Prev: Technical Governance Fellow @pivotal_org 🩵
Ankur Gupta @getpy
37K Followers 3K Following Python Dev, Parent. Author - https://t.co/5lts7q9z7R Curator - https://t.co/wr74oHNs8O Creator - MapToPoster https://t.co/YQt2CoiupJ 🖖
Quentin Lhoest 🤗 @lhoestq
5K Followers 342 Following Datasets & Compute @huggingface since 2020 | Open Source + HF Hub platform
Jason Ho @cod1r
50 Followers 800 Following a nerd for computers aspiring game developer/better programmer 418 I'm a teapotTrends for United States
You might like
Baidu Research
@BaiduResearch
68K Followers
AutoML Conference
@automl_conf
754 Followers
Sander Dieleman
@sedielem
72K Followers
Victor M
@victormustar
33K Followers
Gaurav Aggarwal
@fooobar
17K Followers
Alex Smola
@smolix
22K Followers
Daniel Lemire
@lemire
38K Followers
![[In thick Austrian accent] Just some guy, you know.](https://pbs.twimg.com/profile_images/378800000709778870/cca78522d1f78af68d02505415d8fd2e.jpeg)
Morgan Hough (mhough....
@mhough
812 Followers
Matthew Honnibal
@honnibal
12K Followers
Dimi
@DimitriDeJonghe
3K Followers
Saulnier Lucile
@LucileSaulnier
4K Followers