Rocket U2 | UniVerse & UniData

Hey All,

I'm trying to create a python script which can be called from a BP program (use PyCallFunction or the like) which will log to fluentd.

I have something which works on the first run of the BP program, but crashes on the second call.

To replicate you don't need the BP program, the following will do, from TCL

```

PYTHON

from fluent.sender import FluentSender

sender.emit('uv', { 'level': 'info', 'message': "hello"})

sender.emit('uv', { 'level': 'info', 'message': "world!"})

quit()

PYTHON

from fluent.sender import FluentSender

sender.emit('uv', { 'level': 'info', 'message': "hello"})

```

The error is

```

  File "/home/u2dev/lib64/python3.11/site-packages/msgpack/__init__.py", line 36, in packb
    return Packer(**kwargs).pack(o)
           ^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable

```

It looks like on the second go round msgpack can't import Packer for a .so file.

I've tried setting REINIT.PYTHON OFF, but no luck.

This is on UV 11.4.1 and my python requirements.txt file is contains

fluent-logger==0.11.1
msgpack==1.1.0

I'm hoping one of the bright sparks here can help.

Thanks

Gary

Gary,

I don't know whether this helps, but I found this reference:

msgpack is JSON-like format.
Like JSON, msgpack is not support directly serializing any custom object.
If you want it, you should use pickle.

Regards

JJ

Hey All,

I'm trying to create a python script which can be called from a BP program (use PyCallFunction or the like) which will log to fluentd.

I have something which works on the first run of the BP program, but crashes on the second call.

To replicate you don't need the BP program, the following will do, from TCL

```

PYTHON

from fluent.sender import FluentSender

sender.emit('uv', { 'level': 'info', 'message': "hello"})

sender.emit('uv', { 'level': 'info', 'message': "world!"})

quit()

PYTHON

from fluent.sender import FluentSender

sender.emit('uv', { 'level': 'info', 'message': "hello"})

```

The error is

```

  File "/home/u2dev/lib64/python3.11/site-packages/msgpack/__init__.py", line 36, in packb
    return Packer(**kwargs).pack(o)
           ^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable

```

It looks like on the second go round msgpack can't import Packer for a .so file.

I've tried setting REINIT.PYTHON OFF, but no luck.

This is on UV 11.4.1 and my python requirements.txt file is contains

fluent-logger==0.11.1
msgpack==1.1.0

I'm hoping one of the bright sparks here can help.

Thanks

Gary

I am using the msgpack library indirectly, via fluent-logger, so I don't have much choice to pick a serialization option.

Gary,

I don't know whether this helps, but I found this reference:

msgpack is JSON-like format.
Like JSON, msgpack is not support directly serializing any custom object.
If you want it, you should use pickle.

Regards

JJ

Hey All,

I'm trying to create a python script which can be called from a BP program (use PyCallFunction or the like) which will log to fluentd.

I have something which works on the first run of the BP program, but crashes on the second call.

To replicate you don't need the BP program, the following will do, from TCL

```

PYTHON

from fluent.sender import FluentSender

sender.emit('uv', { 'level': 'info', 'message': "hello"})

sender.emit('uv', { 'level': 'info', 'message': "world!"})

quit()

PYTHON

from fluent.sender import FluentSender

sender.emit('uv', { 'level': 'info', 'message': "hello"})

```

The error is

```

  File "/home/u2dev/lib64/python3.11/site-packages/msgpack/__init__.py", line 36, in packb
    return Packer(**kwargs).pack(o)
           ^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable

```

It looks like on the second go round msgpack can't import Packer for a .so file.

I've tried setting REINIT.PYTHON OFF, but no luck.

This is on UV 11.4.1 and my python requirements.txt file is contains

fluent-logger==0.11.1
msgpack==1.1.0

I'm hoping one of the bright sparks here can help.

Thanks

Gary

Gary,

If msgpack is indeed the root vause and it's not possible to use an alternative, have you thought about launching a PHANTOM to launch the program?

I've seen references to msgpickle as well which supports custom serialisation, but my Python is not up to snuff in this area.

Regards

JJ 

I am using the msgpack library indirectly, via fluent-logger, so I don't have much choice to pick a serialization option.

Gary,

I don't know whether this helps, but I found this reference:

msgpack is JSON-like format.
Like JSON, msgpack is not support directly serializing any custom object.
If you want it, you should use pickle.

Regards

JJ

Hey All,

I'm trying to create a python script which can be called from a BP program (use PyCallFunction or the like) which will log to fluentd.

I have something which works on the first run of the BP program, but crashes on the second call.

To replicate you don't need the BP program, the following will do, from TCL

```

PYTHON

from fluent.sender import FluentSender

sender.emit('uv', { 'level': 'info', 'message': "hello"})

sender.emit('uv', { 'level': 'info', 'message': "world!"})

quit()

PYTHON

from fluent.sender import FluentSender

sender.emit('uv', { 'level': 'info', 'message': "hello"})

```

The error is

```

  File "/home/u2dev/lib64/python3.11/site-packages/msgpack/__init__.py", line 36, in packb
    return Packer(**kwargs).pack(o)
           ^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable

```

It looks like on the second go round msgpack can't import Packer for a .so file.

I've tried setting REINIT.PYTHON OFF, but no luck.

This is on UV 11.4.1 and my python requirements.txt file is contains

fluent-logger==0.11.1
msgpack==1.1.0

I'm hoping one of the bright sparks here can help.

Thanks

Gary

John,

Thanks for your reply.

Can you be more specific about what you mean by a PHANTOM?

The problem I bumped into seams to be a generic u2py problem.

That is it error on run on a import for the second time, where the import is coming from a .so file.

I tried a different approach, instead of using fluent-logger and therefore msgpack, I use a async post request.

Using the aiohttp caused the same error as msgpack, when swapping to the httpx library it worked.

Do anyone have an idea why the following would fail?

"from ._quoting import _Quoter, _Unquoter"

"from ._cmgpack import Packer"

Both imports seems to be from .so files.

Gary,

If msgpack is indeed the root vause and it's not possible to use an alternative, have you thought about launching a PHANTOM to launch the program?

I've seen references to msgpickle as well which supports custom serialisation, but my Python is not up to snuff in this area.

Regards

JJ 

I am using the msgpack library indirectly, via fluent-logger, so I don't have much choice to pick a serialization option.

Gary,

I don't know whether this helps, but I found this reference:

msgpack is JSON-like format.
Like JSON, msgpack is not support directly serializing any custom object.
If you want it, you should use pickle.

Regards

JJ

Hey All,

I'm trying to create a python script which can be called from a BP program (use PyCallFunction or the like) which will log to fluentd.

I have something which works on the first run of the BP program, but crashes on the second call.

To replicate you don't need the BP program, the following will do, from TCL

```

PYTHON

from fluent.sender import FluentSender

sender.emit('uv', { 'level': 'info', 'message': "hello"})

sender.emit('uv', { 'level': 'info', 'message': "world!"})

quit()

PYTHON

from fluent.sender import FluentSender

sender.emit('uv', { 'level': 'info', 'message': "hello"})

```

The error is

```

  File "/home/u2dev/lib64/python3.11/site-packages/msgpack/__init__.py", line 36, in packb
    return Packer(**kwargs).pack(o)
           ^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable

```

It looks like on the second go round msgpack can't import Packer for a .so file.

I've tried setting REINIT.PYTHON OFF, but no luck.

This is on UV 11.4.1 and my python requirements.txt file is contains

fluent-logger==0.11.1
msgpack==1.1.0

I'm hoping one of the bright sparks here can help.

Thanks

Gary