Get model context details from litellm

[caufieldjh]

The litellm package tracks metadata on model context limits in this file: https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json

This may also be retrieved and cached with their helper function:

from litellm import get_model_cost_map
models = get_model_cost_map("")  # This can take a URL but by default it uses the one above

models["gpt-4"]
{'max_tokens': 4096, 'max_input_tokens': 8192, 'max_output_tokens': 4096, 'input_cost_per_token': 3e-05, 'output_cost_per_token': 6e-05, 'litellm_provider': 'openai', 'mode': 'chat', 'supports_function_calling': True}

The list includes multiple model types: completions, embeddings, etc.