Study guide
13.2 Integrating, Re-elaborating & Structuring Data (DigComp 3.2, 3.4)
EPSO Graduate Administrators Open Competition AD5
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Competition Overview, Notice EPSO/AD/427/26 & Two-Part Remote Testing
1.1 Notice EPSO/AD/427/26: Competition Architecture & the Two-Part Testing Model 1.2 Remote Testing on TAO & Proctorio: Setup, Rules & Incident Procedure 1.3 Pacing Strategy, Time Allocation & Scoring Dynamics
2 Chapter 2: Verbal Reasoning: Core Deductive Logic & Text Interpretation
2.1 The Strict Text Boundary Rule 2.2 Formal Deduction, Quantifiers & Conditional Statements 2.3 Identifying & Neutralizing Subtle Distractor Traps
3 Chapter 3: Verbal Reasoning: Advanced Administrative Passages & Pacing
3.1 Navigating Institutional, Regulatory & Policy Texts 3.2 High-Speed Scanning & Structural Mapping Strategies 3.3 Step-by-Step Question Decomposition & Worked Scenarios
4 Chapter 4: Numerical Reasoning: Arithmetic, Data Interpretation & Speed Math
4.1 Essential Calculation Toolkit for EPSO AD5 4.2 Data Interpretation: Tables, Multi-Axis Charts & Complex Graphs 4.3 Mental Estimation, Distractor Elimination & On-Screen Calculator Use
5 Chapter 5: Abstract Reasoning: Sequences, Matrices & Attribute Tracking
5.1 Foundational Geometric Rules & Transformations 5.2 Multi-Attribute Tracking & Dynamic State Changes 5.3 Rapid Systematic Decomposition & The 60-Second Scan
6 Chapter 6: Foundations of EU Law: Treaties, Principles & Legal Sources
6.1 Evolution of the European Treaties 6.2 Constitutional Principles of the European Union 6.3 Hierarchy of EU Legal Acts & General Principles
7 Chapter 7: The Political Institutions: Commission, Council & European Parliament
7.1 The European Commission: Executive & Guardian of Treaties 7.2 The Council of the European Union & The European Council 7.3 The European Parliament: Democratic Oversight & Co-Decision
8 Chapter 8: The Judiciary, Financial Oversight & Advisory Bodies
8.1 The Court of Justice of the European Union (CJEU) 8.2 European Court of Auditors, ECB & Financial Oversight 8.3 Advisory Bodies & Specialized Agencies
9 Chapter 9: EU Law-Making, Delegated Acts & The Multi-Annual Financial Framework
9.1 The Ordinary Legislative Procedure (OLP) 9.2 Special Legislative Procedures & Comitology 9.3 The EU Budget & Multiannual Financial Framework (MFF)
10 Chapter 10: The Internal Market, Competition Policy & Economic Governance
10.1 The Four Freedoms of the Internal Market 10.2 EU Competition Policy & State Aid Control 10.3 Economic and Monetary Union (EMU) & Stability Architecture
11 Chapter 11: EU Strategic Policies: Green Deal, AFSJ, External Action, the 2024–2029 Agenda & Main Spending Policies
11.1 The European Green Deal & Environmental Policy 11.2 Area of Freedom, Security and Justice (AFSJ) 11.3 EU External Action, CFSP & Common Commercial Policy 11.4 The 2024–2029 Agenda: Political Guidelines, Competitiveness, Defence & Simplification 11.5 Cohesion, Agriculture, Social Europe & the Digital Rulebook
12 Chapter 12: DigComp Areas 1 & 2: Information Literacy, Communication & Collaboration
12.1 Information and Data Literacy (DigComp 1.1-1.3) 12.2 Interacting, Sharing, Civic Participation & Collaborating (DigComp 2.1–2.4) 12.3 Netiquette, Digital Identity & Trust Services (DigComp 2.5–2.6)
13 Chapter 13: DigComp Area 3: Digital Content Creation, Copyright & Data Workflows
13.1 Developing Digital Content & Document Processing (DigComp 3.1) 13.2 Integrating, Re-elaborating & Structuring Data (DigComp 3.2, 3.4) 13.3 Copyright, Open Data & Creative Commons Licensing (DigComp 3.3)
14 Chapter 14: DigComp Areas 4 & 5: Cybersecurity, Data Privacy & Technical Problem Solving
14.1 Cybersecurity, Threat Mitigation & Device Safety (DigComp 4.1) 14.2 Data Protection, Well-being & the Environment (DigComp 4.2–4.4) 14.3 Problem Solving, Technical Troubleshooting & Artificial Intelligence (DigComp 5.1-5.4)
15 Chapter 15: The Free-Text Essay on EU Matters (EUFTE): Architecture & Drafting
15.1 The EUFTE: Official Format, Where It Sits & the Five Assessment Anchors 15.2 Document Synthesis & Time-Budgeted Outline Method 15.3 Model Answer Walkthrough & Self-Assessment Against the Anchors
16 Chapter 16: Comprehensive Mock Simulation, Error Remediation & Exam-Day Strategy
16.1 Full Mock Simulation & Endurance Management for Parts 1 and 2 16.2 Systematic Error Log & High-Yield Remediation 16.3 Exam-Day Protocol & From Reserve List to Recruitment
13.2 Integrating, Re-elaborating & Structuring Data (DigComp 3.2, 3.4)
Key Takeaways Explain This Topic AI
DigComp Area 3 encompasses integrating and re-elaborating digital content (3.2) alongside applying computational thinking and programming concepts (3.4) to solve complex public administration challenges.
Structured and semi-structured data interchange in EU workflows depends on CSV (RFC 4180 tabular data), XML (hierarchical schema-validated trees via XSD), and JSON (lightweight key-value and nested array structures for RESTful web services).
Robust data ingestion demands strict handling of character encodings (UTF-8 to prevent multilingual character corruption / mojibake), locale-specific delimiters (commas vs semicolons), and ISO 8601 standardized date-time formatting.
Systematic data cleansing combines deduplication algorithms (exact hash matching and fuzzy Levenshtein distance metrics) with rigorous missing-data imputation strategies (distinguishing MCAR from MNAR).
Computational thinking equips administrators with four cognitive pillars—decomposition, pattern recognition, abstraction, and algorithmic logic—enabling the deployment of low-code ETL pipelines, Robotic Process Automation (RPA), and deterministic workflow rules.
Last updated: September 2026
13.2 Integrating, Re-elaborating & Structuring Data (DigComp 3.2, 3.4)
Official Reference: European Commission Joint Research Centre (JRC) DigComp 2.2 . Competence 3.2 (Integrating and re-elaborating digital content) requires modifying, refining, improving, and integrating new content into existing bodies of knowledge. Competence 3.4 (Programming) establishes the capacity to understand algorithmic logic, design step-by-step instructions for automated systems, and apply computational thinking to solve administrative challenges.
Modern European governance relies on the continuous consolidation of heterogeneous data streams. An AD5 administrator in the European Commission or an EU regulatory agency frequently ingests administrative registers, corporate financial disclosures, and statistical submissions from 27 distinct national administrations. These datasets arrive in divergent formats, utilize varying character encodings, follow disparate date conventions, and contain structural anomalies.
To transform raw inputs into robust evidence for policy impact assessments or comitology deliberations, administrators must possess technical competence in data structures and algorithmic thinking. Misinterpreting data schemas or executing flawed data transformation routines leads to incorrect policy baselines, regulatory compliance failures, and compromised institutional decision-making.
Structured Data Architectures: CSV, XML, and JSON in EU Administration
Public administrations rely on three primary data formats for data storage, inter-institutional exchange, and API communication: CSV , XML , and JSON .
CSV (Comma-Separated Values) & The European Delimiter Dilemma
Governed by RFC 4180 , CSV is a flat, plain-text format representing tabular data where records correspond to rows and fields correspond to columns:
Delimiter Conflicts in Multilingual Europe: In standard Anglo-American computing environments, fields are separated by commas ( , ), and numbers use periods as decimal points (e.g., Brussels,Belgium,1250.75 ). However, in continental European locales (including France, Germany, Italy, and Spain), the comma serves as the statutory decimal separator ( 1250,75 ). Consequently, European datasets typically employ the semicolon ( ; ) or tab ( \t ) as the field delimiter (e.g., Brussels;Belgium;1250,75 ). Attempting to ingest a semicolon-delimited file with a default comma-delimited parser lumps entire rows into a single text string; conversely, parsing a comma-decimal file with a comma delimiter fragments individual numbers into separate columns.
Text Qualifiers: Fields containing delimiter characters, line breaks, or quotation marks must be enclosed within text qualifiers (typically double quotation marks). For example, enclosing text containing commas ensures that embedded commas are not interpreted as column separators.
Character Encoding (UTF-8 vs. Legacy Encodings): European Union administrations handle official documents across 24 official languages utilizing Latin, Greek, and Cyrillic scripts, alongside extensive diacritical marks (e.g., č, ć, ž, é, è, ä, ö, ü, ø, å ). All administrative CSV files must be encoded in UTF-8 (Unicode Transformation Format 8-bit) . Ingesting a UTF-8 file under legacy single-byte encodings (such as Windows-1252 or ISO-8859-1) results in mojibake —the garbled, unreadable rendering of text where multibyte characters decompose into nonsensical strings (e.g., 'Kraków' rendering as 'Kraków' ).
XML (Extensible Markup Language) & Schema Validation
XML represents hierarchical, semi-structured data organized as a tree structure of user-defined elements, tags, and attributes:
Well-Formedness vs. Validity:
Well-Formed XML: A document that satisfies fundamental syntactic rules: contains a single root element, every start-tag has a matching end-tag ( <Country>France</Country> ), tags are strictly nested without overlapping ( <b><i>text</i></b> , not <b><i>text</b></i> ), and attribute values are enclosed in quotation marks ( <Budget year='2026'> ).
Valid XML: A well-formed XML document that additionally conforms to a formal schema definition, such as an XML Schema Definition (XSD) or Document Type Definition (DTD). The schema strictly governs allowed element hierarchies, mandatory versus optional tags, explicit data types (e.g., xs:date , xs:decimal ), string lengths, and cardinality rules ( minOccurs , maxOccurs ).
Namespaces ( xmlns ): XML namespaces prevent element name collisions when integrating data from disparate institutional vocabularies. For instance, <customs:Tariff> can be distinguished from <internal-market:Tariff> within the same consolidated legislative file.
Institutional Applications: XML remains the standard for formal inter-institutional transactions across the European Union, including e-Procurement (e-PRIOR) , the European Single Procurement Document (ESPD) , and SDMX (Statistical Data and Metadata eXchange) used by Eurostat and the European Central Bank (ECB).
JSON (JavaScript Object Notation)
Governed by RFC 8259 , JSON is a lightweight, human-readable, text-based data interchange format based on a subset of JavaScript syntax. It has become the dominant standard for modern web services, public data APIs, and mobile applications.
Core Structural Components:
Objects ( { ... } ): An unordered collection of key-value pairs. Keys must be strings enclosed in double quotation marks, separated from values by colons ( : ), with key-value pairs separated by commas.
Arrays ( [ ... ] ): An ordered collection of values enclosed in square brackets, separated by commas (e.g., an array of Member State codes).
Supported Primitive Data Types: String, Number (integer or floating point), Boolean ( true or false ), and null . Crucially, JSON has no native Date type ; dates are serialized as strings, standardized via ISO 8601.
Strict Syntax Rules: Single quotes are strictly invalid for keys or string values in RFC 8259 JSON; trailing commas after the final element in an object or array are prohibited and trigger parsing syntax errors.
Comparative Technical Matrix
Architectural Dimension CSV (RFC 4180) XML (W3C Standard) JSON (RFC 8259)
Structural Model Flat 2D tabular (rows/columns) Hierarchical tree (nested nodes) Key-value pairs & nested arrays
Schema Validation None native; relies on external validation scripts Rigorous via XSD / DTD schemas Optional via JSON Schema
Data Typing Untyped; all values parsed as text strings Strongly typed via XSD primitive types Native primitive types (string, number, bool, null)
Bandwidth Overhead Extremely low (minimal formatting syntax) High (verbose start/end tags, namespaces) Low to Medium (concise key-value syntax)
Primary EU Use Case Eurostat bulk downloads, raw accounting logs e-Procurement, legal act publishing, SDMX RESTful APIs, Open Data portal endpoints
Loading diagram... Data Cleansing, Transformation and Computational Workflow Architecture
Data Cleansing, Transformation & Re-elaboration
Raw administrative data ingested from external sources is rarely ready for immediate policy modeling. Data cleansing is the systematic process of identifying and correcting corrupt, inaccurate, incomplete, or irrelevant records.
Deduplication Methodologies: Exact vs. Fuzzy Matching
Public registers frequently receive duplicate entries from repeated submissions or synchronized agency databases:
Exact Deduplication: Identifies records that share identical primary keys or identical values across all fields. Exact deduplication is performed efficiently by calculating a cryptographic hash (such as SHA-256 or MD5) over the concatenated record string; identical records generate identical hash digests and are collapsed into a single entry.
Fuzzy Deduplication: Identifies records that refer to the same real-world administrative entity despite typographical errors, differing abbreviations, or transliteration variances (e.g., 'Ministère de l'Économie' versus 'Min. de l'Economie' ).
Levenshtein Distance: The minimum number of single-character edits (insertions, deletions, or substitutions) required to change one word into another. A low edit distance relative to string length indicates a probable duplicate.
Jaro-Winkler Distance: Measures string similarity, assigning higher weight to matching character prefixes, making it particularly effective for administrative organization names and personal surnames.
Missing Data Typology & Imputation Strategies
When statistical datasets contain missing observations, administrators must evaluate the underlying mechanism of missingness before taking remedial action:
Missing Completely at Random (MCAR): Missingness is entirely independent of both observed and unobserved variables (e.g., a laboratory sample tube broke accidentally during environmental testing). Deleting missing records ( Listwise Deletion ) does not introduce systemic bias, though it reduces statistical power.
Missing at Random (MAR): Missingness is related to observed variables but not the missing value itself (e.g., smaller regional administrative units fail to report compliance data due to documented staff shortages, but non-reporting is unrelated to their actual compliance rate).
Missing Not at Random (MNAR): Missingness directly relates to the value of the unobserved variable itself (e.g., high-polluting industrial installations deliberately fail to report carbon emissions). Deleting missing records in MNAR conditions produces severe systemic bias, falsely skewing policy conclusions toward optimal compliance.
Imputation Strategies: Rather than discarding incomplete records, administrators apply imputation:
Mean / Median Imputation: Replacing missing numeric values with the column mean (for normally distributed variables) or median (for skewed administrative variables, such as regional income).
Forward-Fill / Backward-Fill: Propagating the last known value forward in longitudinal time-series data.
K-Nearest Neighbors (KNN) & Model-Based Imputation: Estimating missing values based on records that exhibit similar multi-attribute characteristics.
Text Transformation and Regular Expressions (Regex)
Transforming unstructured citizen consultation responses into structured categories requires text processing and pattern matching using Regular Expressions (Regex) :
Tokenization: Breaking free-text submissions into individual words, phrases, or semantic tokens.
Regex Pattern Matching: Identifying and extracting structured entities from unstructured narrative text using formal pattern syntax:
Extracting CELEX numbers: \b3\d{4}[RLD]\d{4}\b (locates standard EU legislative acts, such as 32024R1183 ).
Extracting email addresses: [a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,} .
Extracting date strings: \d{4}-\d{2}-\d{2} (standard ISO 8601 format).
Date and Time Alignment: The ISO 8601 Standard
Data aggregated across 27 Member States frequently uses incompatible regional date formats. EU data and interoperability guidance recommends the ISO 8601 international standard:
Standard Format: YYYY-MM-DD (e.g., 2026-09-23 ) for calendar dates, and YYYY-MM-DDThh:mm:ssZ for complete date-time stamps.
Ambiguity Elimination: Eliminates dangerous ambiguity between Anglo-American format ( MM/DD/YYYY ) and continental European format ( DD/MM/YYYY ). Under non-standardized formats, 04/05/2026 could represent either April 5 or May 4.
Timezone Offsets: The suffix Z denotes Coordinated Universal Time (UTC). Regional timestamps must explicitly declare their offset from UTC (e.g., +01:00 for Central European Time [CET] and +02:00 for Central European Summer Time [CEST]).
Computational Thinking & Workflow Automation for Public Administrators
DigComp Competence 3.4 requires public servants to apply computational thinking —a structured cognitive methodology for formulating problems and designing deterministic solutions that can be executed by human administrators or automated systems.
The Four Pillars of Computational Thinking
Decomposition: Breaking a complex, overwhelming administrative policy or procedure into smaller, self-contained, manageable sub-problems. For example, decomposing the EU Single Market infringement monitoring process into discrete steps: data ingestion, statutory deadline calculation, formal notice generation, Member State reply logging, and escalation scoring.
Pattern Recognition: Observing trends, recurrences, and commonalities across decomposed components. Identifying that disparate environmental reporting submissions from different Member States share identical underlying validation errors allows the administrator to design a single, reusable transformation script.
Abstraction: Filtering out extraneous, non-essential operational details to focus strictly on the fundamental variables and core mechanics governing the system. When building a policy model for cross-border customs clearance, the administrator abstracts away localized paperwork variations to model clearance times strictly based on cargo type, origin risk score, and inspection capacity.
Algorithm Design: Formulating an ordered, unambiguous, step-by-step sequence of deterministic instructions to solve the problem or achieve an administrative outcome. An algorithm must be finite, precise, and input-output driven.
Workflow Automation Paradigms: Macros, RPA, and ETL Pipelines
Public sector productivity depends on automating repetitive administrative workflows:
Spreadsheet Macros (VBA & Office Scripts): Automated scripts recorded or coded to execute sequential spreadsheet tasks (such as reformatting monthly financial tables, generating standard charts, and compiling regional summaries). Security Consideration: Macros can harbor malicious executable payloads (macro viruses); current Microsoft Office versions block macros in files downloaded from the internet by default, and institutional IT policies typically allow only signed macros from trusted templates.
Robotic Process Automation (RPA): Software 'bots' that emulate human user interactions with administrative graphical user interfaces (GUIs). RPA interacts directly with on-screen buttons, text boxes, and menus to copy data between legacy systems that lack modern APIs (e.g., migrating case file metadata from an older national judicial register into an EU border management database).
Low-Code / No-Code ETL (Extract, Transform, Load) Pipelines: Visual data pipeline tools that automatically extract data from diverse institutional repositories, execute validation and transformation rules (filtering, joining, normalizing), and load clean, structured datasets into centralized data warehouses without requiring low-level programmatic coding.
Conditional Logic and Boolean Decision Trees
Administrative rules are inherently logical. Translating policy directives into automated administrative workflows requires mastery of Boolean logic :
Logical Operators:
AND (Conjunction): Evaluates to TRUE only if all component conditions are satisfied.
OR (Disjunction): Evaluates to TRUE if at least one component condition is satisfied.
NOT (Negation): Inverts the truth value of the condition ( TRUE becomes FALSE ).
XOR (Exclusive OR): Evaluates to TRUE if exactly one condition is true, but FALSE if both are true or both are false.
Operator Precedence: In complex logical statements, the evaluation hierarchy follows: NOT takes precedence first, followed by AND , with OR evaluated last. Parentheses must be deployed to override default precedence and ensure unambiguous execution.
Short-Circuit Evaluation: In modern automated logical engines, evaluation stops as soon as the final truth value is determined: in an AND expression, if the first condition is FALSE , subsequent conditions are not evaluated; in an OR expression, if the first condition is TRUE , remaining conditions are bypassed. Administrators leverage short-circuit logic to prevent processing errors (such as dividing by zero or querying a null database field).