English Manual

1. Overview

PyWPEM is a Python toolkit for whole-pattern X-ray diffraction decomposition and structure refinement. It treats the measured diffraction pattern as a physics-constrained probabilistic mixture and updates peak positions, peak shapes, peak weights, and lattice constants through EM iterations coupled with Bragg-law consistency.

The usual entry point is from PyXplore import WPEM. The main user-facing functions are:

• WPEM.BackgroundFit: fit and subtract spectral background.
• WPEM.CIFpreprocess: read CIF files and generate HKL, 2θ, multiplicity, and extinction information.
• WPEM.XRDfit: run whole-pattern WPEM fitting, decomposition, and lattice refinement.
• WPEM.Plot_Components: visualize decomposed phase or peak components.
• WPEM.XRDSimulation: simulate XRD patterns from CIF files.
• WPEM.SubstitutionalSearch: search substitutional solid-solution configurations.
• WPEM.Amorphous_fit and WPEM.AmorphousRDFun: fit amorphous halos and compute RDF.
• WPEM.XPSfit: decompose XPS main and satellite peaks through EM.

2. Installation

Detailed tutorial: Class 1 Installation

An isolated conda environment is recommended. The tutorials use Python 3.10.

conda create -n pywpem python=3.10
conda activate pywpem
pip install notebook
pip install PyXplore
pip install --upgrade PyXplore

Start Jupyter Notebook:

jupyter notebook

Test the installation in a notebook or Python script:

from PyXplore import WPEM

3. Data Preparation

Detailed tutorial: Class 2 Single-phase refinement

3.1 Experimental Pattern

XRD input is normally a two-column file without a header. The first column is 2theta; the second column is intensity.

10.000, 123.4
10.010, 125.1
10.020, 128.7

Read a CSV file:

import pandas as pd
intensity_csv = pd.read_csv("intensity.csv", header=None)

For in situ or instrument-exported Excel files:

intensity_excel = pd.read_excel("intensity.xlsx", header=None)

3.2 CIF Files

Prepare one CIF file for each candidate crystalline phase. CIFpreprocess reads lattice constants, atom coordinates, space-group information, and symmetry operations, then writes the HKL files used by the fitting step.

latt, atom_coordinates, density = WPEM.CIFpreprocess(
    filepath="Mn2O3.cif",
    two_theta_range=(15, 75)
)

3.3 Working Directory

By default, PyWPEM writes folders such as ConvertedDocuments, output_xrd, and WPEMFittingResults in the current directory. For batch processing, use a separate work_dir for every sample to prevent output files from overwriting each other.

4. Single-Phase XRD Refinement

Detailed tutorial: Class 2 Single-phase refinement

This workflow follows the Mn2O3 single-phase example and is the recommended first complete run.

Step 1: Import and Load Data

from PyXplore import WPEM
import pandas as pd

intensity_csv = pd.read_csv("intensity.csv", header=None)

Step 2: Fit Background

var = WPEM.BackgroundFit(
    intensity_csv,
    lowAngleRange=17,
    poly_n=13,
    bac_split=16,
    bac_num=300
)

This step writes ConvertedDocuments/no_bac_intensity.csv and ConvertedDocuments/bac.csv. If you run inside the data folder, you can pass the generated file names directly. If you use work_dir, pass full paths.

Step 3: Preprocess CIF

latt, atom_coordinates, density = WPEM.CIFpreprocess(
    filepath="Mn2O3.cif",
    two_theta_range=(15, 75)
)

latt is the initial lattice vector in [a, b, c, alpha, beta, gamma] format. density can be used later for mass-fraction estimation.

Step 4: Run WPEM Refinement

wavelength = [1.540593, 1.544414]
Lattice_constants = [latt]

WPEM.XRDfit(
    wavelength,
    var,
    Lattice_constants,
    "no_bac_intensity.csv",
    "intensity.csv",
    "bac.csv",
    subset_number=11,
    low_bound=20,
    up_bound=70,
    bta=0.85,
    iter_max=5,
    asy_C=0,
    InitializationEpoch=0
)

Important Parameters

• wavelength: X-ray wavelengths. Cu Kα1/Kα2 commonly uses [1.540593, 1.544414].
• subset_number: number of peaks used for Bragg lattice updates.
• low_bound and up_bound: 2θ range for selecting Bragg-update peaks.
• bta: Lorentzian fraction in the pseudo-Voigt peak shape.
• iter_max: maximum iterations. Use a small number for testing and a larger value for production analysis.
• InitializationEpoch: number of initialization epochs with frozen peak positions.

5. Multiphase XRD Refinement

Detailed tutorial: Class 3 Multiphase refinement

The multiphase workflow is similar to the single-phase workflow, except that every candidate phase must be preprocessed and the lattice constants are passed as a list.

from PyXplore import WPEM
import pandas as pd

intensity_csv = pd.read_csv("intensity.csv", header=None)
Var = WPEM.BackgroundFit(
    intensity_csv,
    LFctg=0.2,
    lowAngleRange=20,
    window_length=7,
    bac_num=500
)

NaCl, _, density_nacl = WPEM.CIFpreprocess("NaCl.cif", two_theta_range=(11, 110))
LiCO, _, density_lico = WPEM.CIFpreprocess("Li2CO3.cif", two_theta_range=(11, 110))

wavelength = [1.540593, 1.544414]
WPEM.XRDfit(
    wavelength,
    Var,
    [NaCl, LiCO],
    "no_bac_intensity.csv",
    "intensity.csv",
    "bac.csv",
    subset_number=11,
    low_bound=20,
    up_bound=50,
    bta=0.9,
    iter_max=5,
    asy_C=0,
    InitializationEpoch=2,
    num=7,
    Ave_Waves=True
)

WPEM.Plot_Components(
    lowboundary=10,
    upboundary=110,
    wavelength=wavelength,
    name=["NaCl", "Li2CO3"],
    phase=2
)

phase=2 tells the plotting function to draw two crystalline phases. Keep name in the same order as Lattice_constants.

6. XRD Simulation

Detailed tutorial: Class 4 XRD simulations

XRDSimulation generates simulated XRD patterns from CIF files. It is useful for phase validation and synthetic-data generation.

FHKL_square, two_theta_sim, intensity_sim = WPEM.XRDSimulation(
    filepath="PSO.cif",
    two_theta_range=(10, 120, 0.01),
    bacI=True,
    GrainSize=30,
    orientation=[-0.1, 0.1],
    thermo_vib=0.1,
    zero_shift=0.01
)

• two_theta_range=(start, stop, step) controls angular range and resolution.
• GrainSize models peak broadening caused by crystallite size.
• orientation models intensity perturbation from preferred orientation.
• thermo_vib models thermal displacement of average atomic positions.
• zero_shift models instrumental zero shift.
• bacI=True adds a random polynomial background.

7. In Situ XRD Batch Processing

Detailed tutorial: Class 5 In situ XRD

In situ or operando XRD often contains many sequential patterns. Use a separate temporary folder for every pattern, copy the input and required peak0.csv into that folder, and pass it through work_dir.

import os
import shutil
import pandas as pd
from PyXplore import WPEM

wavelength = [1.540593, 1.544414]
Lattice_constants = [[2.87549, 2.87549, 14.02056, 90.0, 90.0, 120.0]]
results = {}
num_samples = 5

for i in range(num_samples):
    idx = i + 1
    tmp_dir = f"tmp/{idx}"
    os.makedirs(tmp_dir, exist_ok=True)

    tmp_intensity = os.path.join(tmp_dir, "intensity.xlsx")
    shutil.copyfile(f"data/{idx}.xlsx", tmp_intensity)
    shutil.copyfile("peak0.csv", os.path.join(tmp_dir, "peak0.csv"))

    intensity_excel = pd.read_excel(tmp_intensity, header=None)
    var = WPEM.BackgroundFit(
        intensity_excel,
        lowAngleRange=17,
        LFctg=0.0,
        poly_n=8,
        bac_split=10,
        bac_num=80,
        work_dir=tmp_dir
    )

    no_bac_file = os.path.join(tmp_dir, "ConvertedDocuments", "no_bac_intensity.csv")
    bac_file = os.path.join(tmp_dir, "ConvertedDocuments", "bac.csv")

    duration, lattice = WPEM.XRDfit(
        wavelength,
        var,
        Lattice_constants,
        no_bac_file,
        tmp_intensity,
        bac_file,
        Ave_Waves=True,
        subset_number=6,
        low_bound=15,
        up_bound=60,
        bta=0.85,
        iter_max=3,
        asy_C=0,
        InitializationEpoch=0,
        work_dir=tmp_dir
    )
    results[idx] = lattice

After the loop, save the lattice parameters to CSV or Excel and plot them against time, temperature, voltage, or cycle number.

8. Solid-Solution Search

Detailed tutorial: Class 6 Solid solution searching

A typical solid-solution workflow first performs XRD refinement, then uses the WPEM output of one phase to search candidate substitutional structures.

WPEM.SubstitutionalSearch(
    xrd_pattern="CrystalSystem0_WPEMout_2026.6.10_11.1.csv",
    cif_file="Mn2O3.cif",
    random_num=20,
    wavelength="CuKa",
    search_cap=20,
    SolventAtom="Mn3+",
    SoluteAtom="Ru2+",
    max_iter=15,
    cal_extinction=False
)

After searching, use XRDSimulation to simulate candidate structures and compare them with the experimental or decomposed pattern.

9. Amorphous Analysis

Detailed tutorial: Class 7 Amorphous study

For semicrystalline or amorphous-rich samples, first decompose crystalline peaks with XRDfit, then fit the remaining broad halo with Amorphous_fit, and optionally compute RDF.

wavelength = [1.03]
Lattice_constants = [[17.53, 17.53, 6.47, 90, 90, 120]]

WPEM.XRDfit(
    wavelength,
    var,
    Lattice_constants,
    "no_bac_intensity.csv",
    "intensity.csv",
    "bac.csv",
    subset_number=3,
    low_bound=6,
    up_bound=16,
    bta=0.78,
    iter_max=10,
    asy_C=0,
    InitializationEpoch=0
)

WPEM.Amorphous_fit(
    mix_component=2,
    sigma2_coef=0.5,
    max_iter=200,
    peak_location=None,
    Wavelength=1.03
)

WPEM.Plot_Components(
    lowboundary=4,
    upboundary=19,
    wavelength=wavelength,
    Macromolecule=True,
    phase=1
)

WPEM.AmorphousRDFun(
    wavelength=1.03,
    r_max=4,
    density_zero=None,
    Nf2=1,
    highlight=6
)

mix_component is the number of amorphous broad-peak components. If approximate positions are known, use peak_location=[20, 30, "fixed"] to fix peak centers.

10. XPS Decomposition

Detailed tutorial: Class 8 X-Ray spectrograph

PyWPEM includes an EM-based XPS decomposition module. The input is also two columns: binding energy and intensity. Fit the background in XPS mode, then define main peaks and satellite peaks.

from PyXplore import WPEM
import pandas as pd

intensity_csv = pd.read_csv("CuO.csv", header=None)
var = WPEM.BackgroundFit(
    intensity_csv,
    segement=[[910, 931], [948, 952], [958, 959], [966, 970]],
    bac_num=120,
    Model="XPS",
    noise=0.05,
    bac_var_type="multivariate gaussian"
)

AtomIdentifier = [
    ["CuII", "2p3/2", 933.7],
    ["CuII", "2p1/2", 954]
]

satellitePeaks = [
    ["CuII", "2p3/2", 941.6],
    ["CuII", "2p3/2", 943.4],
    ["CuII", "2p1/2", 962.5]
]

WPEM.XPSfit(
    var,
    AtomIdentifier,
    satellitePeaks,
    "no_bac_intensity.csv",
    "CuO.csv",
    "bac.csv",
    bta=0.80,
    iter_max=50
)

AtomIdentifier defines main peaks in [chemical state, orbital, initial binding energy] format. satellitePeaks defines satellite peaks.

11. Output Files

12. Troubleshooting

Input files have different lengths

XRDfit requires the original pattern, background-subtracted pattern, and background pattern to have the same number of points. Re-run BackgroundFit and make sure you pass files generated from the same folder.

All peak positions are shifted

Check wavelength and instrumental zero shift. Try ZeroShift=True for fitting or use zero_shift in simulation to understand the direction of the offset.

Multiphase outputs are hard to interpret

Keep Lattice_constants, density_list, name, and phase output order consistent.

Batch outputs are overwritten

Use a separate work_dir for every sample. The in situ workflow uses folders such as tmp/1 and tmp/2 for this reason.

Background fitting is unsatisfactory

Adjust LFctg, lowAngleRange, bac_split, bac_num, and poly_n. For XPS, use segement to manually specify background regions.