-
-
Notifications
You must be signed in to change notification settings - Fork 98
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Mamba (minimal) #918
Closed
Closed
Add Mamba (minimal) #918
Commits on Jan 26, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 5c532ec - Browse repository at this point
Copy the full SHA 5c532ecView commit details -
Configuration menu - View commit details
-
Copy full SHA for fb91f13 - Browse repository at this point
Copy the full SHA fb91f13View commit details
Commits on Feb 1, 2024
-
Update safetensors module and naming
- Makes the safetensors module private. - Doesn't get exported on the preamble, avoiding a naming clash with the safetensors external crate. - Change how and when the period is inserted. - This should make it closer to how the fields are accessed in the code.
Configuration menu - View commit details
-
Copy full SHA for a832f51 - Browse repository at this point
Copy the full SHA a832f51View commit details -
Configuration menu - View commit details
-
Copy full SHA for 901cfe4 - Browse repository at this point
Copy the full SHA 901cfe4View commit details -
Configuration menu - View commit details
-
Copy full SHA for a14b40b - Browse repository at this point
Copy the full SHA a14b40bView commit details -
- Add the try_normalize_rms related functions. - Add the `LayerRMSNorm1D` module.
Configuration menu - View commit details
-
Copy full SHA for b52932c - Browse repository at this point
Copy the full SHA b52932cView commit details -
- Add `TrySplitShapeAlong` and `TrySplitTensorAlong`. - Minor linting and docs fix. TODO - Check if the tape should be returned. If not, it can be removed from the interface. - Add cuda kernel. - Consider a different interface, where it could get split in more than two tensors - possibly stated on a vec. In this way it could get closer to the pytorch interface (chunks).
Configuration menu - View commit details
-
Copy full SHA for 693b699 - Browse repository at this point
Copy the full SHA 693b699View commit details -
Configuration menu - View commit details
-
Copy full SHA for de55567 - Browse repository at this point
Copy the full SHA de55567View commit details
Commits on Feb 2, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 3122f78 - Browse repository at this point
Copy the full SHA 3122f78View commit details -
Configuration menu - View commit details
-
Copy full SHA for ace3808 - Browse repository at this point
Copy the full SHA ace3808View commit details -
Configuration menu - View commit details
-
Copy full SHA for f6d06e0 - Browse repository at this point
Copy the full SHA f6d06e0View commit details
Commits on Feb 6, 2024
-
- Also added `from_fn` for Arrays. Note: the interface currently requires two passes for construction, one for creating a list of tensors with NoneTape and another for putting tapes into those tensors.
Configuration menu - View commit details
-
Copy full SHA for ea424c3 - Browse repository at this point
Copy the full SHA ea424c3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5994ac5 - Browse repository at this point
Copy the full SHA 5994ac5View commit details
Commits on Feb 7, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 24a8593 - Browse repository at this point
Copy the full SHA 24a8593View commit details
Commits on Feb 8, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 5ffff2d - Browse repository at this point
Copy the full SHA 5ffff2dView commit details
Commits on Feb 9, 2024
-
Added {load/read/save/write}_safetensor_with methods
This alternative method: - Requires load/read to decide whether it should skip missing tensors; - Requires load/read/save/write to decide how should keys be mapped.
Configuration menu - View commit details
-
Copy full SHA for e883b28 - Browse repository at this point
Copy the full SHA e883b28View commit details -
Configuration menu - View commit details
-
Copy full SHA for c695a15 - Browse repository at this point
Copy the full SHA c695a15View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4141e06 - Browse repository at this point
Copy the full SHA 4141e06View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8202b20 - Browse repository at this point
Copy the full SHA 8202b20View commit details -
Configuration menu - View commit details
-
Copy full SHA for 34234e2 - Browse repository at this point
Copy the full SHA 34234e2View commit details
Commits on Feb 20, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 93202ad - Browse repository at this point
Copy the full SHA 93202adView commit details -
Configuration menu - View commit details
-
Copy full SHA for eb70a88 - Browse repository at this point
Copy the full SHA eb70a88View commit details -
Configuration menu - View commit details
-
Copy full SHA for fde7a40 - Browse repository at this point
Copy the full SHA fde7a40View commit details -
Configuration menu - View commit details
-
Copy full SHA for 75d63cd - Browse repository at this point
Copy the full SHA 75d63cdView commit details -
Configuration menu - View commit details
-
Copy full SHA for f0bcb9a - Browse repository at this point
Copy the full SHA f0bcb9aView commit details -
- Add stateless forward impl. - Efficient for training (but training is not yet implemented). - Input requires the entire sequence, and requires no state cache. - Generates one output for each input sequence. - Add stateful forward impl. - Efficient for inference. - Input requires the last single sequence point, and requires the last state cache. - Generates a single output referring to the last input.
Configuration menu - View commit details
-
Copy full SHA for cac2f33 - Browse repository at this point
Copy the full SHA cac2f33View commit details -
Configuration menu - View commit details
-
Copy full SHA for bff1b65 - Browse repository at this point
Copy the full SHA bff1b65View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.