Tag Archive for: MapForce

How to Create Batch Data Mapping Projects


A common requirement in data processing is batch data mapping, especially in the context of data transformation and integration. It involves converting data in batches rather than processing individual data points one at a time. Batch data mapping is often required in data integration scenarios where input from multiple sources needs to be aligned or transformed together. Two common scenarios are “batch to batch” and “batch to one.”

In our latest series of MapForce demo videos, we explore these common data mapping challenges.

Decorative image: yellow piping in a factory setting

BATCH TO BATCH DATA MAPPING

Batch to batch data mapping is helpful in scenarios where you have data updates or changes coming in batches, and you need to synchronize or transform these batches together. This could be to convert them to a different format, perform some type of sort or calculation, or a combination of these.

In this demo, we create a data mapping project that reads files from a directory and uses wildcards to set up a mapping that will process data from multiple files at once. Then, we explore another option for defining batch conversion using dynamic file names supplied by the mapping. This demo also shows how to add calculations and comments to your data mapping projects.

While this video highlights a batch to batch transformation of JSON files to XML files, MapForce supports conversion and transformation for any combination of XML, JSON, PDF, database, text, Protobuf, Excel, XBRL, and so on, for advanced data integration and ETL processes.

BATCH TO ONE DATA TRANSFORMATION

Batch to one data transformation is another common requirement, for example, when you want to merge or combine multiple files into a single consolidated document and perform some data transformation, conversion, or calculation operations in between.

This example also explores multiple approaches to defining the batch process, which will be applicable in different scenarios:

  1. Specifying a collection of files in the same directory using a wildcard
  2. Selecting batch files based on a list of file names stored in a different directory

This demo also shows how to sort the data merged from multiple files before writing it to the target.

After watching these quick demos, you can download a free trial of MapForce to try batch mapping, conversion, and transformation for yourself.

MORE MAPFORCE DEMOS

If you liked these videos, check out the rest of the MapForce demo series, which covers everything from mapping XML and JSON to databases to configuring data processing functions and extracting data from PDF documents.

Tags: , , , , ,

AI-based Database Image Classification with Altova MapForce


One of the most common examples of AI in our everyday lives is facial recognition. Facial recognition is the process of identifying or verifying a person’s identity based on their face. Facial recognition is used in many applications, such as unlocking our phones with FaceID, tagging our friends on social media platforms like Facebook, and checking in at airports or hotels with biometric scanners. Facial recognition can make our lives more convenient and secure, but it can also raise some privacy and ethical concerns. For instance, how can we ensure that our facial data is not misused or stolen by hackers or malicious actors? How can we prevent facial recognition from being used for surveillance or discrimination? How can we ensure that facial recognition is accurate and fair, and does not have any biases or errors?

The paragraph above was generated by ChatGPT in response to my request to describe the benefits and risks of artificial intelligence and include a real-life example. It’s interesting that ChatGPT chose FaceID as the example, since FaceID is simply one variation of image analysis and AI-powered image classification offers potential to automate many real-world tasks.

One common use-case is a product catalog, wherein a company manages product information provided by many different manufacturers. A product loaded into that database may have a name that does not necessarily include a precise description of the item. For instance, wellington is a boot, fedora is a hat, a mongoose is a bicycle, and a yellow watermelon shiny needlefish is a fishing lure. We can make use of AI-powered image classification using the Microsoft Azure Cognitive Services Computer Vision API to address this problem. The Computer Vision Service takes the image data or URL as its input and returns information about the content. One service generates image classification tags based on a training set of recognizable objects, living beings, scenery, and actions that the Azure AI has been trained on. These tags allow us to categorize products in the database accordingly and may even correspond to search terms a user might provide to find products in the catalog.

decorative image depicting an AI "brain"
Read more…
Tags: , , , ,

AI-based support request sentiment analysis using MapForce and GPT-4


Automated sentiment analysis of text, such as user reviews, has historically been a challenge. Because of the myriad intricacies of natural language, systems faced difficulties in analyzing context and nuances. This required an inordinate amount of manual work to overcome.

One of the many useful capabilities of modern AI systems that are based on large language models (LLMs) such as OpenAI’s GPT-4 is that they are very good at sentiment analysis of natural text inputs. We can use that capability to build a very efficient database solution in MapForce that, for example, goes through all the new incoming records in a support database and automatically determines whether a particular support request or other customer feedback is positive, negative, constitutes a bug report, or should be considered as a feature request.

Decorative lead photo depicting machine learning for sentiment analysis
Read more…
Tags: , , , ,

How to Create a Chained Data Transformation


Data mapping plays a vital role in modern data-driven organizations, enabling efficient data management and integration. Altova MapForce is a powerful, graphical data mapping tool that supports endless data transformation scenarios, including one-to-one, one-to-many, many-to-one, and chained data conversion.

While there are applications for each of these approaches, chained data mapping is especially helpful for complex data processing tasks where multiple stages of data manipulation are required. Here’s a look at the benefits of a chained data conversion approach – and a video of how MapForce makes the process easy and straightforward.

Decorative image: three business people working on laptops
Read more…
Tags: , , , ,

How to Convert Data in MapForce [Video]


Altova MapForce offers a powerful, yet easy-to-use, approach to converting and transforming data. Whether you need to write XML to a database, convert JSON to EDI, or map Excel to multiple different data formats, MapForce has you covered.

From simple one-to-one conversions to complex ETL scenarios, the MapForce approach is to represent data structures as graphical components. To associate fields, drag and drop connecting lines. A comprehensive library of data filters and functions is available for transforming data before writing it to the target.

Decorative image showing a person working on a laptop with data formats supported by Altova MapForce in the background

We have recently revamped our series of Introduction to MapForce videos. Each short how-to gives a demo of a common MapForce scenario.  

Start at the beginning to learn how MapForce works:

And follow along to learn:

You can follow along with the examples in these how-to videos by downloading a free, 30-day trial of MapForce. Check back for new MapForce videos, which are added often.

Tags: , , , , , , ,

Validating and Debugging Data Transformations


Software developers and other data professionals often need to transform data from one format to another. These transformations can be simple one-to-one conversions or may require more complex manipulation. For instance, relationships must be generated when importing flat CSV files into a database, or source data may need to be split for the target, as in full name vs. first, middle, last, and optional suffix. Validating data transformation is critical to prevent data loss or corruption.

In an earlier post on Web service data integration, we combined a string value for GMT time with a numeric offset in seconds to generate the local time for weather forecasts. We created a user function that performed all the steps required to complete this operation. MapForce includes a powerful interactive data mapping debugger that can easily trace and validate this transformation. Let’s take a look at how it works.

A software developer working on a computer
Read more…
Tags: , , , ,

Transforming and Converting Protobuf


MapForce supports mapping protocol buffers (Protobuf) to and from other structured data formats as mapping sources or targets. In the constant quest for more efficient ways to transfer, manipulate, and manage large structured data sets, Google has created a language- and platform-neutral data format similar to XML, but smaller, faster, and simpler than even JSON data. Tools are available to generate and work with Protobuf using Java, Python, C++, C#, Ruby, and other programming languages.

The structure of any Protobuf message is defined in a .proto file that defines each field name and value type. Altova MapForce lets users drop these .proto files into a data mapping as a source or target along with any other data, including XML, JSON, relational databases, Excel, flat files, REST and SOAP web services, and others.  .proto files versions 2 and 3 are supported.

A MapForce data mapping creates compatibility between existing XML, JSON, database or legacy data formats and new applications leveraging the efficiency of Protobuf.

Read more…

Tags: , , , ,

Excel Data Mapping to Update Existing Documents


Excel began life as a simple spreadsheet tool. Over time, support for rich text styling options, built-in charts, and copy and paste formatting features has led many enterprises to create reports in Excel documents. This can cause difficulty when data changes and existing documents need to be manually updated for distribution to a wide audience in the familiar report style.

Altova MapForce, the award-winning, graphical data mapping tool for any-to-any conversion and integration, supports Excel data mapping to convert data to existing Excel documents while preserving styles and formulas in the original.

This feature lets you write directly to nicely formatted Excel files to update data at runtime: any designated worksheets, rows, and cells from the specified file will be replaced with data from the mapping and all formatting in the existing file will be preserved as-is. To protect functionality in the existing spreadsheet, cells with formulas are not overwritten.

Let’s look at an example of how to map Excel data.

Financial pros using XBRL
Read more…
Tags: , , ,

Data Mapping NoSQL Databases


NoSQL databases are non-tabular databases that store data differently than traditional databases made up of relational tables. Two of the most popular NoSQL databases, MongoDB and Apache CouchDB, store data as collections of BSON (binary JSON) and JSON documents. These databases leverage flexible JSON schemas and scale easily with large amounts of data and high user loads.

Altova MapForce has long supported data mapping all popular relational databases and now also includes native support for data mapping NoSQL databases. MapForce includes functionality for inserting, extracting, filtering, and ordering NoSQL data. Let’s look at an example.

Read more…
Tags: , , ,

NoSQL Database Support and More in Version 2022


Altova Software Version 2022 is now available, with exciting new support for mapping and converting NoSQL databases in MapForce, pure text report output in StyleVision, and Windows 11 across the product line. The release also adds support for the exciting new OIM standard from XBRL International.

Here’s a look at the highlights.

Read more…
Tags: , , , , , ,

Tools for JSON Comments and JSON Lines


Altova XMLSpy and MapForce JSON tools have long supported JSON and JSON5 for editing as well as data mapping and conversion. As new JSON formats arise in response to real-world usage, the support in these tools is expanding.

This article will help explain the advantages of two newer formats –  JSON Comments and JSON Lines – and show how to use them in XMLSpy and MapForce.

Tools for JSON Comments and JSON Lines
Read more…
Tags: , , , , , ,

API Data Mapping


Web service data integration with MapForce is a popular and proven strategy to capture timely information for analysis or generation of user-friendly reports. In an earlier post we demonstrated API data mapping in 5-day weather forecasts for busy cargo shipping ports by reading Web service data in JSON format and mapping to richly formatted Excel spreadsheets. The weather API we used  is hosted by OpenWeather, a provider of historical, current, and weather forecast data.

But integrating data from any API is not a set-it-and-forget-it task. When you build a solution based on external data, you have to react quickly when the data structure changes. Since our original integration project OpenWeather revised the data delivered by their API. The API now includes wind gust predictions in a JSON property separate from wind speed. Since wind gusts are suspected as a cause of the recent Suez Canal blockage the new data is very relevant to our application! Fortunately, both the MapForce data mapping and the Excel spreadsheet are easily revised to add new data.

Read more…
Tags: , , , ,

Data Mapping with XSLT3 Math Functions


XSLT3 adds trigonometry and other advanced math functions, new formatting functions, functions to collect environment variables, and more, extending XSLT and XSLT2 XML transformation standards. Data analysts and other data professionals can apply XSLT3 functions to solve XML data mapping and integration challenges that require complex mathematical computations. Let’s look at some MapForce examples of data mapping with XSLT3 math functions using trigonometry and other complex math expressions.

Read more…
Tags: , , ,

New XML Grid and More in v2021r2


In the latest release of Altova desktop developer tools and server software products, we’re introducing a completely rebuilt XML Grid View, support for XSLT3 for XML data mapping, statistics and charts for monitoring FlowForce Server, and much more. Let’s take a look at the highlights of Altova Software Version 2021 Release 2. 

New features in Altova v2021r2
Read more…
Tags: , , , , , ,

Data Mapping Binary Objects – Part 2


Binary objects – BLOBs — can be cumbersome to manage in databases. In an earlier post we described a MapForce data mapping to insert binary objects into a database with generated metadata to identify the BLOBs later. The companion challenge in data mapping binary objects is to extract binary data and save it in a comprehensible form faithful to the original.

Let’s look at how that’s done.

Read more…
Tags: , ,

New tools for JSON, EDI, SWIFT, and XBRL in Version 2021


As standards-based technologies and their applications evolve, so does the functionality that users require from developer tools. In our latest release, we’ve added new ways to work with XBRL, JSON, EDI, and more that help Altova customers work faster and more efficiently.

Let’s take a look at some highlights from the Altova Software Version 2021 release.

Read more…
Tags: , , , , , , ,

Web Service Data Integration


In a previous post we wrote that every data integration and reporting task needs to start with a clear understanding of the source data. Using grid view in XMLSpy, the industry-leading XML and JSON editor, we analyzed JSON data for 5-day weather forecasts retrieved from a Web service.

Continuing with our earlier scenario, we’ll use MapForce, the award-winning, graphical data mapping tool for any-to-any conversion and integration, to map the forecasts for a series of major cargo shipping ports into nicely formatted Excel documents. We’ll want to highlight any predicted high winds or heavy rainfall that could cause delays by interfering with cranes loading and unloading containers, or slowing ships entering and exiting the harbors.

Read more…
Tags: , , , , ,

New Data Integration Tools


Altova MissionKit tools offer numerous ways to connect to, query, and integrate data from disparate sources. With multiple product releases each year, we’re constantly working to deliver increased power and efficiency for data integration, while adding features requested by customers. This includes ongoing updates to built-in support for all major SQL databases across the product line.

Let’s take a look at some of the recently added tools and enhancements.

New data integration tools in Altova's release
Read more…
Tags: , , , , , , ,

How to Convert Legacy Text Files [Video]


It’s a common requirement to convert non-standard or legacy text files to or from structured data formats like XML, JSON, and relational databases. However, many times legacy text files are not in a format that can be readily processed by data mapping tools, especially when they have a complex and unique structure that does not consistently fit into CSV or fixed-length field patterns. Moreover, sometimes you need to extract only portions of useful data from a legacy text file.

MapForce, Altova’s any-to-any data conversion tool, includes a unique utility called FlexText that makes it easy to visually define templates for parsing text files and making them accessible to the data mapping tool.

See how FlexText works in our video tutorial.

The example files referenced in the video are available here and you can try FlexText with a free, 30-day trial of MapForce.

Tags: , ,

Data Mapping JSON Lines


The JSON data format continues to evolve as an open standard as it is creatively applied to new data interchange requirements. JSON Lines, defined at http://jsonlines.org/, is a convenient text format for storing structured data where each record is a single line and a valid JSON object. JSON Lines handles tabular data and clearly identifies data types without ambiguity. This allows records to be processed one at a time, which makes the format very useful for exporting and sending data.

Altova MapForce supports data mapping JSON Lines as either a data source or target. Let’s look at a mapping project to extract records from a database table and map to a JSON Lines file for output.

Read more…
Tags: , , , ,

CbC Reporting Made Easy


A recent mandate from the OECD called on large, multinational companies to report financials annually for each country in which they do business to their local tax authority. The OECD specified that this detailed Country by Country (CbC) Report be filed in an XML document according to their reporting schema. But for tax departments that work largely in Excel or other accounting software, this presented a significant stumbling block – and companies found themselves scrambling to meet the requirements by the deadline.

What was needed is a way to automatically generate valid, properly formatted CbC XML reports based on existing data. Altova created the Country by Country Reporting Solution to do just that, either based on manually entered data or figures imported directly from Excel. Let’s take a look at how it works.

 

Read more…

Tags: , , , ,

Transitioning Data Mapping Projects from Development through Testing and Production


Data mapping projects often mirror software development efforts with distinct phases for design, testing, and deployment. This is especially true for ETL (Extract Transform Load) projects when repeated data mapping execution is required as new data becomes available, and the stakes increase higher with large data sets. The Altova MissionKit and Server Software products provide Global Resources to define configurations for each project phase and smoothly transition between them.

Let’s take a look at an example based on a MapForce data mapping from a source file to a database.

Read more…
Tags: , , , , ,

Handle HTTP Errors During Automated Data Integration


Data analysts and other professionals often need to generate real-time data through automated execution of data mappings that request Web services and save the results. During automated execution it’s important to gracefully handle any unexpected HTTP error rather than terminate the integration task.

In an earlier post we discussed conditional processing of a REST Web service response to handle HTTP errors, where separate output files were generated for a normal response and an error. Now let’s look at a revised mapping solution for the airport status example to generate a single mapping result file that contains either the requested airport status or a description of the error.

Read more…
Tags: , , , , ,

Handling HTTP Errors in Web Service Data Mappings


Data integration projects that include information from external Web services may be vulnerable to HTTP errors when retrieving remote data. When data mappings run under automated control it’s especially important to detect and report errors even if errors only occur very rarely.

A MapForce data mapping can include Web service calls and output the result directly to a file or database, or combine it with other inputs for further processing. Regardless of the final output, an HTTP Web service error encountered in a REST Web service request puts the mapping at risk.

MapForce includes features for handling HTTP errors instead of simply aborting execution of a mapping. Developers can configure the body of a REST Web service call to handle and report exceptions based on the HTTP status code returned.

Let’s look at an example.

Read more…
Tags: , , , , ,

Database Mapping with Database Exception Handling


Critical business processes depend on reliable data and database administrators and other data analysts want to be confident in the integrity of information stored in database tables. During automated ETL (Extract Transform Load) operations or other database import tasks, invalid data might be encountered that jeopardizes success of the procedure. Altova MapForce includes database exception handling to roll back the affected data when an error occurs and optionally proceed with the rest of a database mapping.

For instance, an error in a single record need not prevent execution of a mapping from continuing, such as when certain database constraints prevent the mapping from inserting or updating invalid data.

Read more…
Tags: , , ,

Database Tracing to Log Changes Made by a Data Mapping Project


Database administrators and other data professionals often want to maintain a record of changes in critical databases, especially when updates are made by automated scripts or other operations. Database tracing lets administrators track critical changes or anomalies, and help recover from errors. Altova MapForce supports database tracing for all popular relational databases to log the changes made by a data mapping project to the database when the mapping runs.

When tracing is enabled, events such as database insert or update actions, or errors, are logged in an XML file that you can later analyze or process further in an automated way.

Database tracing can be enabled at the database component, table, stored procedure, or database field level. You can choose to trace all messages or only errors, or you can disable tracing completely.

In addition to tracing errors that occur during the execution of a mapping to a target database, MapForce also enables database transaction handling to roll back the affected part of the database data when an error occurs, then optionally proceed with the rest of the mapping.

Read more…
Tags: , , , ,

Mapping Structured Data with Enhanced Node Functions


We’ve reported previously on support for node functions that simplify mapping structured data by eliminating need to copy-paste a function multiple times into a mapping. Repeating the same function unnecessarily clutters the mapping layout and makes the data mapping more difficult to understand or revise.

Now in MapForce 2019, additional filters are available for defining node functions. These new parameters allow developers to apply functions and default values to specific nodes based on custom-defined criteria. For example, you can apply a node function based on node metadata such as the node name, node length, precision of the node’s data type, customized node annotations, and more.

Let’s look at a mapping with enhanced node functions.

Read more…

Tags: , , ,

Get Sharp with Altova’s Latest Release


Altova Software Version 2019 introduces over 20 new features to help you sharpen your development game – starting with support for high-res monitors in both XMLSpy and UModel. There are also tools for working with new standards and database versions across the product line, the ability to map and convert data in Google Protocol Buffers format, and much more. Let’s take a look at the highlights.

Altova Version 2019

Read more…

Tags: , , , , , , ,

Node Functions Simplify Mapping Hierarchical Data Structures


MapForce node functions simplify mapping hierarchical data such as XML nodes or CSV, JSON, EDI, or database fields by permitting users to define a data processing function at the node level and apply it recursively to all descendant items.

Similarly, default values can also be assigned to nodes and automatically applied to descendants.

Defaults and node functions are particularly useful when a data mapping and transformation task requires the same processing logic for multiple descendant items in a structure, for example:

  • Replace null values with some other value, recursively for all descendant items
  • Replace a specific value (for example, “N/A”) with some other value recursively for all descendant items
  • Replace all database null values when reading from a database table
  • Trim all trailing spaces for all values from a source database
  • Append a custom prefix or suffix to all values written to a target file or database
  • Formatting of output values
  • And many others

Defaults and node functions simplify mapping hierarchical data by eliminating need to copy-paste the same function multiple times into a mapping. Repeating the same function unnecessarily clutters the mapping layout and makes it more difficult to understand or revise.

Let’s look at an example.

Read more…

Tags: , ,

The Only JSON Dev Tool You’ll Need


While XMLSpy might not be the first tool developers think of when they’ve got a JSON development task, XMLSpy includes comprehensive support for working with JSON, JSON Schema, and related technologies.

Over the past few product releases, we’ve added intelligent functionality for editing and converting JSON and JSON5 data to the product. We’ve completed the circle with one-click conversion between XML Schemas and JSON Schemas, as well as sample instance generation and JSON Schema documentation generation. And, most recently, we’ve added support for processing JSON with XSLT,  XPath, and XQuery.

Let’s walk through some common examples demonstrating this functionality – and see how these time-saving tools make XMLSpy the only JSON development tool you’ll need.

Developer using JSON tool

Read more…

Tags: , , , ,

Data Mapping NCPDP SCRIPT


EDI (Electronic Data Interchange) standards allow participants with different roles in an industry to communicate clearly and rapidly, and date back to the earliest implementations of electronic communication in the 1950s, long before modern business technologies such as ERP, CRM, and many others. Yet even today, EDI standards continue to evolve to support new requirements and opportunities.

MapForce has long supported data mapping to and from ANSI X12, UN/EDIFACT and other popular EDI standards, and now in the latest release adds support for data mapping NCPDP SCRIPT.

SCRIPT is the state of the art EDI standard developed by the National Council for Prescription Drug Programs (NCPDP) for electronically transmitting medical prescriptions, also known as ePrescribing (eRX) in the United States.

Read more…

Tags: , , ,

Add to Your Collection of Development Building Blocks


When it comes to developing sophisticated software and data integration applications, the more building blocks a developer has at his or her disposal, the better. With each release of our developer and server software, we aim to provide customers with unique tools that give them the edge to get the job done – faster, and more efficiently.

To that end, Version 2018 delivers a unique HTTP testing window, 3-way file comparison functionality, support for SQL Templates, super-powered new server options, and much more.

Read more…

Tags: , , ,

Support for JSON5 in Altova MissionKit, Server Products, and MobileTogether


Altova products have supported JSON for several years. Now, Version 2017 Release 3 of MissionKit and Server products, and MobileTogether Version 3.2 all include support for JSON5 across the product line.

The JSON data format was originally designed to be machine-written and consumed, promoting efficient communication between servers. Usage has expanded and JSON5 is a proposed extension intended to make JSON code easier for humans to write and read.  JSON5 extends JSON by adding some ECMAScript 5 features and, like JSON, is a strict subset of JavaScript. Specifically, JSON5 permits inline and block comments, allows long strings to be split over several lines, and defines alternate legal syntax options for quotes and commas.  These features are not permitted in standard JSON, so files containing the proposed enhancements are typically identified with the .json5 filename suffix.

This post details specific support for JSON5 in each Altova product.

Learn about JSON5 support in Altova tools

Read more…

Tags: , , , , , ,

Use Join to Integrate Data in Any Format


Join is a powerful SQL operation implemented across most database types and familiar to database users. Join is typically used to select and combine information from multiple database tables.

Altova MapForce includes a join component for data mapping that works like a SQL join for database tables and extends data integration functionality by empowering users to join data trees of any data format. Anyone familiar with join operations for database tables will find the MapForce join component especially intuitive. A join operation in MapForce can even combine two different data formats and produce output in a new format altogether.

Read more…

Tags: , , ,

New XSLT Back-mapping Headlines Altova Release


It’s time for the latest release of Altova desktop developer tools and server software products, and this one introduces numerous innovative features across the product line, including a brand-new version of MapForce Server called MapForce Server Accelerator Edition for even faster processing of data integration jobs.

Let’s take a look at the highlights of Version 2017 Release 3.

Read more…

Tags: , , , , ,

A Typical MapForce Server Use Case


Envision a manufacturing company that controls costs by exploiting a just-in-time assembly process with a very low supply of parts inventory on hand. New customer orders are logged in a sales database and at the end of every day the components needed to assemble that day’s sales are tabulated.

The IT department runs a SQL query to identify the required parts and transforms the list into a purchase order in JSON format to be transmitted to the supply chain.

Sound familiar? Our recent blog series on JSON tools and JSON data mapping were based on this real-life scenario. In this post we describe a MapForce Server use case that automates the repetitive task of generating each day’s purchase order.

Read more…

Tags: , , ,

JSON Data Mapping and Transformation with MapForce


JSON is a popular format for transferring data between systems thanks to its simple markup, small footprint, and heritage based on the JavaScript programming language. MapForce supports JSON as both an input and output format for JSON data mapping and transformation. For instance, MapForce can extract information from any popular database and produce a JSON file ready for transfer.
The Requirement: Here is an example of a typical need for JSON data mapping: A manufacturing company controls costs by exploiting a just-in-time assembly process with very little parts inventory on hand. New customer orders are logged in a sales database, and at the end of every day the components needed to assemble that day’s sales are tabulated via a query into the database. The required parts will be ordered from suppliers via a purchase order transferred in JSON format.

Read more…

Tags: , , ,

EDI Data Mapping with MapForce


Any computer industry standard that promotes reliability and clear communication between independent enterprises will have a long life. EDI (Electronic Data Interchange) originated in the 1960s to enable automated transactions between corporate computer systems. EDI remains in widespread use today and continues to evolve to meet modern requirements, under the administration of the UN/EDIFACT and ANSI standards bodies.

Altova MapForce supports EDI data mapping between EDI messages and XML, JSON, relational databases, flat files, Excel, or other data formats to bridge between commonly used information interchange and in-house technologies.

MapForce includes support for the latest EDIFACT versions 2015B and 2016A including the new VERMAS message. Mapping and translating EDIFACT messages to other usable data types for transfer, storage, and management is a common business requirement solved by MapForce.

container_ship1

Read more…

Tags: , , , ,

MapForce Supports SQL Merge When It’s the Right Tool for the Job


Large database tables can easily contain a million, even hundreds of millions of rows of data. Database administrators and others charged with maintaining such large datasets are always concerned about execution time for ETL (Extract, Transform, and Load) operations, updates, and other SQL queries. To make these operations more efficient, some — but not all — database vendors implemented a SQL merge statement to insert or update rows of an existing table as a single bulk-insert statement rather than requiring individual statements for each row.

Altova MapForce automatically supports SQL merge when it is available for the target database. Let’s look at an example.

shutterstock_66084286

Read more…

Tags: , , ,

MapForce Offers Dynamic Access to Node Names


There are situations, especially when encountering loosely structured data, where you may want to map and transform structural components of a data stream along with content. MapForce 2017 includes a new feature to dynamically access node names of XML elements, attributes, or text file columns such as the contents of CSV files, to target components.

Dynamic access to node names allows creation on the fly of target elements and attributes whose names do not need to be known beforehand or specifically identified in the data mapping. This feature lets you create much more generic, flexible, and reusable mappings that require less manual intervention if data models evolve.

News about Dynamic Access to Node Names in MapForce 2017

Read more…

Tags: , , ,

Big Data, Database, and XBRL Tools Get a Huge Boost


It’s time for the latest release of Altova’s MissionKit and server software products, and this one delivers a huge boost in functionality for working with big data, databases, XBRL, and much more.

With support for Apache Avro in multiple products, additional databases and drivers across the product line, and two new XBRL specifications in developer and server products, plus a new way to build and process PDF forms, Version 2017 connects all the data dots.

What will be your favorite new feature? Let’s take a look.

Big Data Support

Read more…

Tags: , , , , , , , ,

Faster EDI Data Translation


Electronic Data Interchange (EDI) standards accelerate commerce worldwide by allowing companies and other organizations – even those in different regions, using different languages and currencies – to send and receive unambiguous information. Some EDI communication protocols and message formats still in wide use today were developed over 30 years ago, when telecommunication systems were slower and data storage was more expensive.

119647393_700px

Read more…

Tags: , , ,

XBRL Data Mapping: WIP Taxonomy


The usefulness of the XBRL standard reaches far beyond SEC requirements for filing financial statements. Organizations such as XBRL.US and XBRL International are working to develop XBRL taxonomies and accompanying software solutions for countless other practical applications where standardizing data submission results in increased accuracy and productivity for all involved – for report filing, data analysis, and beyond.

One such project is the Work in Progress (WIP) Taxonomy created by XBRL.US for the surety industry. The new taxonomy helps save time and increase accuracy for report submission, and at the same time enables new opportunities for data analysis and decision making.

Construction-site-surety

Read more…

Tags: , ,

Data Mapping REST Web Services


MapForce 2016 Release 2 includes expanded functionality for Web Services data mapping, providing robust support for REST Web services. MapForce accepts XML or JSON as the Web service response, allows definition of parameters, and supports custom HTTP headers. Users may define the Web service interface manually or by importing settings from a WADL file or a URL. Manual definition of REST Web Service Settings lets developers create settings based on a template URL. This is a convenient step when developers test and refine REST calls in a Web browser window, since the URL can be copied from the browser to become the template.

REST Web Services can be a pipeline of information for a data mapping project

Read more…

Tags: , , , ,

Applying Data Mapping Patterns


Altova MapForce includes powerful mapping components that correspond to design patterns for data transformation requirements. Analyzing a data mapping challenge up front and following a few straightforward guidelines can uncover data mapping patterns that help simplify creation of the mapping design and lead to an optimal solution. The MapForce Examples project provides sample mapping files and data sets that illustrate many common data mapping patterns. Reviewing these examples and executing them with the MapForce Built-in Execution Engine is another good way to help select the best pattern for your own project.

shutterstock_240957604

Read more…

Tags: , ,

Interactive Debugger for Data Integration Projects


MapForce 2016 introduces a revolutionary data mapping debugger that lets developers working on data integration projects examine data mapping output step by step to diagnose and perfect projects of any complexity. The MapForce data mapping debugger gives users deep insight into the exact inner workings of data integration and ETL projects in a way that was never before possible.

The debugger works with all MapForce data mappings for any combination of XML, XBRL, JSON, databases, flat files, EDI, Excel, or Web services data, including chained mappings, mappings with multiple input or output components, and mappings that include user-defined functions.

Data Mapping Debugger

The MapForce data mapping debugger supports breakpoints and conditional breakpoints, and includes multiple manual stepping options to manually debug a data mapping or continue execution after a breakpoint is reached, allowing users to see as much detail as they need.
Read more…

Tags: , ,

Data Mapping Files with XML Mixed Content


As more specialized data is adapted for delivery on demand to mobile devices with smaller screens, documents with complex structures can be a challenge. Fortunately, MapForce, Altova’s graphical data mapping, conversion, and integration tool, has special features to accelerate data integration projects of all sizes.

For example, richly-formatted documents may include XML mixed content, a complex datatype that permits an XML element to contain text, attributes, and other child elements. Common instances of XML mixed content include blocks of text with tags for bold or italic fonts, or tags that select terms for indexing or other special treatment. Documents with XML mixed content can be difficult to include in a data integration project, since they appear to be free-form and unpredictable, with child elements that occur in any order or even multiple times within the parent.

Mapping_mixed_content

Read more…

Tags: , ,