
Inspirationsconsulting
Add a review FollowVisión general
-
Seleccionar Máquinas y HP
-
Empleos publicados 0
-
(Visto) 7
Descripción de la compañía
Open-R1: a Completely Open Reproduction Of DeepSeek-R1
Hey there! This article is an introduction to the task, not a claim that we have actually reproduced R1 yet. We’re integrating in the open, so as quickly as we have examination numbers, we’ll share them. You can follow our development on Hugging Face and GitHub.
True, but it appears like there’s absolutely nothing to be assessed since right now. I assume the supreme objective is to train a new reasoning design and then use the very same examination metrics as o1 and the DeepSeek-R1.
Well, there need to be at least some sanity check and validation to guarantee the model was trained properly.
Oh yes, if you are speaking about the examination number of deepseek’s model it’s coming soon!
As discussed in the blog post there is no design called Open-R1 to test at all … not yet anyway. This is a blog describing that Hugging face will take the R1 Deepseek model, exercise how it was built as detailed in the paper and from what they released, and then reproduce that process.
in fact this is quite much how science works … A creates a strategy, discovery or development and it is tested by B, C and D to see if it is reproduceable. Thats been the foundation of research study now for a few centuries.
This blog site is not stating they have actually currently done so … Its a blog site describing an intent to begin training a design like R1 and calling it Open-R1.
Also DeepSeek-R1 was just launched recently, and even in their paper they described the calculate hours needed. While those are low calculate hours for a SOTA design this does not mean you can train stated design in a week. I ‘d personally like to be able to train a transformer design in a week, but we may require to wait a while for that level of calculate technology.
So there are no standards for a model that has not been built yet right? As laid out in the blog site, and again in reply to your question.
However fear not, there is a GitHub Repo already and factors (hell I may join myself), some prelim work done, and a strategy of attack. A great beginning position.
n
@edbeeching
has actually assessed the launched designs currently
( src: https://x.com/edwardbeeching/status/1884273209136275742)
R1 just trained on o1 outputs, so jointly …/ s. This is what the brand-new AI czars are saying
Hi! This blog post is an intro to the task, not a claim that we’ve reproduced R1 yet. We will absolutely share the missing out on piece when we have them, you can anticipate the designs and datasets to be upload in this Hugging Face org and the code to be in this GitHub repo
That’s good and important to understand this remarkable hype that lacks technical understanding and explanation. Science has to do with reproduction, and if they declare to be open, let them fullfill the open part.
Please do release the training expense.
We will!
Excalidraw Hi n
@bojan2501
thanks, we will certainly be working hard to ensure this training recipe can work for little language designs on consumer hardware considering that not everybody has a cluster of H100s at home:-RRB- The tool we used for the images was Excalidraw! https://excalidraw.com
looking forward to it! WTF are your discussing?
must be a joke
It’s actually cool to see how the entire open source community comes together!
Ops …
5.5 M is number press reporter in the deepseekv3 tech report (just the training, not the experiment afaik), for R1 tough to approximate tbh but much less than 5.5 M imo
Historically, they have never ever launched code or datasets of their LLM training, so I wouldn’t expect this time to be various. If they would release it that would be remarkable naturally!
Yes of course!
So basically you’re asking to change existing censorship with another flavour of censorship?
The code for the designs are inside the design repositories, e.g. for V3: https://huggingface.co/deepseek-ai/DeepSeek-V3/blob/main/modeling_deepseek.py
Hello Team, I’m Ray Bernard, the author and developer of EQUATOR. My research team will be working on a paper focused on reproducing certain parts of DeepSeek R1. Our objective is to replicate the cold start and provide your group with a dataset that includes COT and other techniques to support these efforts. We like to contribute our work to help. Please let me know if you discover this useful. Best, Ray Bernard https://www.facebook.com/groups/1186310571520299/
Where is the evaluation numbers? without it you can’t call it reproduction.
8 replies
True, however it seems like there’s nothing to be evaluated since today. I assume the ultimate goal is to train a brand-new thinking design and after that use the very same evaluation metrics as o1 and the DeepSeek-R1.
That’s rather fascinating, I was asking myself why the questions the author exposed here are not being asked by others? I believe the work they have done is however at the exact same time I wonder why they wouldn’t put these missing out on pieces on if they are supposed to be fully open.
Why even without reproduction and comprehension of the innovation they could affect a lot the marketplace in this method?
4 replies
Hi! This blog site post is an intro to the task, not a claim that we’ve replicated R1 yet. We will totally share the missing piece when we have them, you can expect the designs and datasets to be upload in this Hugging Face org and the code to be in this GitHub repo
Interesting read, and it is excellent that we see more effort into this instructions: more optimization and less strength.
Also question what tool did the author use for developing step diagram.
2 replies
Excalidraw I’m so grateful that initiative like this already exist, I’m gon na attempt to contribute:-RRB- 1 reply
eagerly anticipating it! So racist articel
2 replies
WTF are your talking about?
Awesome to have this open reproduction began!
For Step # 1 check out https://github.com/open-thoughts/open-thoughts!
https://x.com/ryanmart3n/status/1884284101265612856
Let’s do this thing!
1 reply
It’s really cool to see how the entire open source neighborhood comes together!
Does anybody know the actual training expense of r1? I can’t discover it in the paper or the announcement post. Is the 6M cost reported by media simply the number taken from v3’s training cost?
2 replies
Ops …
Has anybody asked the DeepSeek group to publish their training information and code, or at least share them privately with an independent replication job like this? Have they turned down such a demand?
A devoted replication depends upon using the same dataset and hyperparameters. Otherwise, any major inconsistencies with the released standards would be difficult to pin down-whether due to training data differences or the replication technique itself.
1 reply
Historically, they have never launched code or datasets of their LLM training, so I wouldn’t anticipate this time to be various. If they would release it that would be amazing obviously!
In the meantime we have to make best guess estimates and see if we can get there ourselves.
You offer good replication process of Deepseek reasoning training. I will attempt something similar to it.
This is actually excellent information, can we fine tune with particular use case when code is launched?
1 reply
Yes of course!
Please think about getting rid of biased, polluted or unaligned training data and make an effort to remove copyrighted works from the crawl from consumption. This will make the design more usable. If you recycled anthropic curation checks, this might likewise assist, get rid of obviouslybiased information will likely add a great deal of value. We do not want another polluted, unaligned open source model, right? And no business would ever utilize deepseek or a design that recycles it, right?
We appreciate your work for the benefit of humankind, we hope.
Miike C from NJ
1 reply
So basically you’re asking to change existing censorship with another flavour of censorship?
Can’t wait! Hopefully the model will be uncensored however whatever you can do is alright! Love seeing open source building itself up. I’m not wise adequate to in fact assist however I can contribute support lol
Hello guys, I am even simply looking for code for DeepSeek-V2, in order to fully comprehend multi-head hidden attention. You do not seem to have code in Hugging Face even for that. Or am I missing out on something? Don’t see anything in src/transformers/models. MLA is not properly described in their paper, so it would be necessary to have code for this.