Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decouple bytes generator from XtcePacketDefinition #158

Open
medley56 opened this issue Mar 7, 2025 · 1 comment
Open

Decouple bytes generator from XtcePacketDefinition #158

medley56 opened this issue Mar 7, 2025 · 1 comment
Labels
enhancement New feature or request

Comments

@medley56
Copy link
Member

medley56 commented Mar 7, 2025

Description

We have two separate jobs that are currently being mashed together in the packet definition object.

  1. Generating bytes objects (or subclasses of bytes) that represent 1 packet of binary data per object
  2. Parsing those bytes objects into dictionary-like Packet objects

The XtcePacketDefinition object contains a packet_generator method that conflates these two responsibilities.

Implementation Plan

Pull the packet_generator out of XtcePacketDefinition, so without further abstraction, the packet parsing process becomes:

definition = XtcePacketDefinition(xml_file)
packet_generator: Iterator[bytes] = custom_bytes_yielding_generator_implementation(binary_file)
packets: list[Packet] = [definition.parse_packet(Packet(bytes_data=generated_bytes) for generated_bytes in packet_generator] 

This can easily be further abstracted to eliminate the need for users to provide their own generator if they are using CCSDS, but it allows users to generate packet bytes objects that are non-CCSDS packets, but still represented by XTCE.

@medley56 medley56 added the enhancement New feature or request label Mar 7, 2025
@greglucas
Copy link
Collaborator

I think we can extend this even further to remove the need for a separate generator. So we could still have definition.packet_generator() yield packets, but they would be packets that the definition itself determined not any underlying generator.

I'm not sure we would be able to accept any readable source though, I think the parsing would need the full packet's worth of bytes available to parse because there is no wait-to-read within the parsing methods (that is all currently handled at the ccsds-generator level). So my thought would be something like (with some additional UnrecognizedPacket error handling and the like).

def packet_generator(binary_data: bytes):
    # outer loop's parsing position
    parsing_pos = 0
    while parsing_pos < len(binary_data) * 8
        # Put a reference to all the binary data into our packet and
        # then update the current parsing position
        packet = Packet(binary_data=binary_data)
        packet._parsing_pos = parsing_pos
        packet = self.parse_packet(packet)
        # Trim the binary data held within the packet?
        packet.binary_data = binary_data[parsing_pos // 8 : packet._parsing_pos // 8]
        # Advance the outer loop's parsing position with what we just read
        parsing_pos = packet._parsing_pos
        yield packet

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants