OpenCompass v0.1.8

🔥 Highlights

• 🌐 New Dataset Integrations: Expanding our dataset collection with Tabmwp, py150, maxmin, and more.
• 💡 Compatibility and API Support: Enhancements with MiniGPT-4 and MiniMax API, and support for Xunfei API.
• 🛠️ Local Environment and Debugging Improvements: Streamlined local debugging and usage of datasets from local paths.

🚀 New Features & Enhancements

• Datasets Galore: Unleash the power of new datasets including Tabmwp, py150, maxmin, and updates to existing ones like Mathbench for broader research scope (#505, #546, #562).
• MiniGPT-4 & MiniMax API Compatibility: Stay up-to-date with the latest versions and extended API support (#539, #548).
• Xunfei API Model & Update: Explore new possibilities with the integration and update of Xunfei API (#547, #572).

🛠 Improvements & Fixes

• Local Debug Mode Restriction: Enhanced resource management in local debug mode (#522 by @yingfhu).
• Various Fixes and Updates: Addressing typos, import issues, and log redirections for smoother operation (#520, #549, #551, #555, #564).

📚 Documentation Updates

• Enhanced README and FAQs: Get all your queries answered and understand OpenCompass better with updated documentation (#523, #531, #535, #540, #567).
• Typo Corrections: Ensuring clarity and accuracy in our documentation (#530, #533).

🎊 New Contributors

A warm welcome to the new members of the OpenCompass community!

• @Sanster, @ayushrakesh, @HimanshuMahto, @shresthasurav, @bittersweet1999, and @jingmingzhuo. Thank you for your valuable contributions!

Changelog

• add multi model viz by @Sanster in #509
• fix typo in WSC prompt by @Sanster in #520
• [Fix] fix local debug mode not restrict the resources by @yingfhu in #522
• Update README.md - one enhancement. by @ayushrakesh in #523
• Typo error in README.md by @HimanshuMahto in #531
• docs: fix typos in markdown files by @shresthasurav in #530
• [Doc] Update README and FAQ by @tonysy in #535
• [fFeat] Add an opensource dataset Tabmwp by @bittersweet1999 in #505
• [Feature]: To be compatible with the latest version of MiniGPT-4 by @YuanLiuuuuuu in #539
• [Doc] Update README by @tonysy in #540
• [Feat] support xunfei api model by @yingfhu in #547
• [Feature] Add support for MiniMax API by @tonysy in #548
• 【Feature】Update Mathbench dataset prompt and fix small errors by @liushz in #546
• [Fix] fix filename typo by @yingfhu in #549
• [Feat] support cidataset by @yingfhu in #538
• [Fix] fix registry error with internal by @yingfhu in #551
• [Fix] fix unnecessary import and update requirements by @yingfhu in #555
• [Fix] fix log re-direct by @yingfhu in #564
• Add py150 and maxmin by @jingmingzhuo in #562
• [Doc] Update api.txt by @tonysy in #567
• [Docs] add humanevalx dataset link in config by @yingfhu in #559
• [Docs] fix GLUE_CoLA dataset name error by @KevinNuNu in #533
• [Feature] Update xunfei api by @tonysy in #572
• [Feature] Add CMB zero-shot evaluation by @Leymore in #571
• [Feature] Use dataset in local path by @Leymore in #570
• [Sync] update model configs by @Leymore in #574
• [Sync] Bump version to 0.1.8 by @Leymore in #576

Explore the detailed changes in the full changelog.

Thank you to everyone who contributed to this release. Your efforts are immensely appreciated and are helping to make OpenCompass a more robust and versatile tool. Let's continue to push the boundaries with OpenCompass v0.1.8! 🚀🌐🛠️