Logs for deployments

It seems like logs are limited to 10k lines for deployments. Is there a way to send logs somewhere else or to tail it? It doesn’t seem like the CLI supports viewing logs from deployments.

This seems pretty crucial or else it’s impossible to debug what’s going on when there are errors

Hi James,

We have extended the log limit to 100k lines and we plan on extending this even further. Imposing a limit is only temporary. In the meantime, you may use workarounds, such as papertrail. You would just need to modify your logging middleware to send an HTTP request for each log or bulk event like in the Token-based destination section.

1 Like

Hi, @James_Yu - one more quick update. We’ve updated our Python package to allow you to download and interact with Deployments logs. For example,

>>> from gradient import gradient_deployments
>>> import json
>>>
>>>
>>> logs = gradient_deployments.get_deployment_logs(
...     deployment_id='6753a6cc-393a-478d-bc29-9e8828a0aa2f',
...     limit=5,
...     api_key=os.getenv('PS_API_KEY'))
>>> print(json.dumps(logs, indent=4))
[
    [
        2,
        "",
        "2021-12-13T21:41:39.711Z"
    ],
    [
        3,
        "  Welcome to Streamlit. Check out our demo in your browser.",
        "2021-12-13T21:41:39.711Z"
    ],
    [
        4,
        "",
        "2021-12-13T21:41:39.711Z"
    ],
    [
        5,
        "  Network URL: http://10.42.40.230:8501",
        "2021-12-13T21:41:39.711Z"
    ],
    [
        6,
        "  External URL: http://172.83.13.4:8501",
        "2021-12-13T21:41:39.711Z"
    ]
]

or, using a generator,

>>> logs = gradient_deployments.yield_deployment_logs(
...     deployment_id='6753a6cc-393a-478d-bc29-9e8828a0aa2f',
...     limit=5,
...     api_key=os.getenv('PS_API_KEY'))
>>> next(logs)
LogRow(line=2, message='', timestamp='2021-12-13T21:41:39.711Z')
1 Like

Following up on this, I believe we’ve since also modified our logs in the web console to tail the most recent N log lines instead of always displaying the first N

1 Like