Cross-Language Serialization

This page explains how to use Fory for cross-language serialization between C++ and other languages.

Overview

Apache Fory™ enables seamless data exchange between C++, Java, Python, Go, Rust, and JavaScript. The xlang (cross-language) mode ensures binary compatibility across all supported languages.

Enabling Cross-Language Mode

#include "fory/serialization/fory.h"

using namespace fory::serialization;

auto fory = Fory::builder()
    .xlang(true)  // Enable cross-language mode
    .build();

Cross-Language Example

C++ Producer

#include "fory/serialization/fory.h"
#include <fstream>

using namespace fory::serialization;

struct Message {
  std::string topic;
  int64_t timestamp;
  std::map<std::string, std::string> headers;
  std::vector<uint8_t> payload;

  bool operator==(const Message &other) const {
    return topic == other.topic && timestamp == other.timestamp &&
           headers == other.headers && payload == other.payload;
  }
};
FORY_STRUCT(Message, topic, timestamp, headers, payload);

int main() {
  auto fory = Fory::builder().xlang(true).build();
  fory.register_struct<Message>(100);

  Message msg{
      "events.user",
      1699999999000,
      {{"content-type", "application/json"}},
      {'h', 'e', 'l', 'l', 'o'}
  };

  auto result = fory.serialize(msg);
  if (result.ok()) {
    auto bytes = std::move(result).value();
    // Write to file, send over network, etc.
    std::ofstream file("message.bin", std::ios::binary);
    file.write(reinterpret_cast<const char*>(bytes.data()), bytes.size());
  }
  return 0;
}

Java Consumer

import org.apache.fory.Fory;
import org.apache.fory.config.Language;

public class Message {
    public String topic;
    public long timestamp;
    public Map<String, String> headers;
    public byte[] payload;
}

public class Consumer {
    public static void main(String[] args) throws Exception {
        Fory fory = Fory.builder()
            .withLanguage(Language.XLANG)
            .build();
        fory.register(Message.class, 100);  // Same ID as C++

        byte[] bytes = Files.readAllBytes(Path.of("message.bin"));
        Message msg = (Message) fory.deserialize(bytes);

        System.out.println("Topic: " + msg.topic);
        System.out.println("Timestamp: " + msg.timestamp);
    }
}

Python Consumer

import pyfory

class Message:
    topic: str
    timestamp: int
    headers: dict[str, str]
    payload: bytes

fory = pyfory.Fory()
fory.register(Message, type_id=100)  # Same ID as C++

with open("message.bin", "rb") as f:
    data = f.read()

msg = fory.deserialize(data)
print(f"Topic: {msg.topic}")
print(f"Timestamp: {msg.timestamp}")

Type Mapping

Primitive Types

C++ Type	Java Type	Python Type	Go Type	Rust Type
bool	boolean	bool	bool	bool
int8_t	byte	int	int8	i8
int16_t	short	int	int16	i16
int32_t	int	int	int32	i32
int64_t	long	int	int64	i64
float	float	float	float32	f32
double	double	float	float64	f64

String Types

C++ Type	Java Type	Python Type	Go Type	Rust Type
std::string	String	str	string	String

Collection Types

C++ Type	Java Type	Python Type	Go Type
std::vector<T>	List<T>	list	[]T
std::set<T>	Set<T>	set	map[T]struct{}
std::map<K,V>	Map<K,V>	dict	map[K]V

Temporal Types

C++ Type	Java Type	Python Type	Go Type
Timestamp	Instant	datetime	time.Time
Duration	Duration	timedelta	time.Duration
LocalDate	LocalDate	datetime.date	time.Time

Field Order Requirements

Critical: Field will be sorted by their snake_cased field name, converted name must be considten across langauges

C++

struct Person {
  std::string name;   // Field 0
  int32_t age;        // Field 1
  std::string email;  // Field 2
};
FORY_STRUCT(Person, name, age, email);  // Order matters!

Java

public class Person {
    public String name;   // Field 0
    public int age;       // Field 1
    public String email;  // Field 2
}

Python

class Person:
    name: str    # Field 0
    age: int     # Field 1
    email: str   # Field 2

Type ID Consistency

All languages must use the same type IDs:

// C++
fory.register_struct<Person>(100);
fory.register_struct<Address>(101);
fory.register_struct<Order>(102);

// Java
fory.register(Person.class, 100);
fory.register(Address.class, 101);
fory.register(Order.class, 102);

# Python
fory.register(Person, type_id=100)
fory.register(Address, type_id=101)
fory.register(Order, type_id=102)

Compatible Mode

For schema evolution across language boundaries:

// C++ with compatible mode
auto fory = Fory::builder()
    .xlang(true)
    .compatible(true)  // Enable schema evolution
    .build();

Compatible mode allows:

• Adding new fields (with defaults)
• Removing unused fields
• Reordering fields

Troubleshooting

Type Mismatch Errors

Error: Type mismatch: expected 100, got 101

Solution: Ensure type IDs match across all languages.

Encoding Errors

Error: Invalid UTF-8 sequence

Solution: Ensure strings are valid UTF-8 in all languages.

Related Topics

• Configuration - Builder options
• Type Registration - Registering types
• Supported Types - Type compatibility