-
Oct 20, 2023 | 
hackernoon.com | Jiafeng Zhang
This post is about building a unified OLAP platform. An insurance company tries to build a data warehouse that can undertake all their customer-facing, analyst-facing, and management-facing data analysis workloads. The main tasks include:Self-service insurance contract query: This is for insurance customers to check their contract details by their contract ID. It should also support filters such as coverage period, insurance types, and claim amount.
-
Oct 9, 2023 | 
hackernoon.com | Jiafeng Zhang
HackerNoon Mobile App (ahhhhh) - now trending #147 on Apple news 🔥New! Pixelated Icons Open Sourced by HackerNoonWrite on Future of AI to win $1000! 🔥🔥🔥You can now get notified about hottest tech job openings in our Noonifications Newsletters.
-
Sep 20, 2023 | 
hackernoon.com | Jiafeng Zhang
What is Apache Doris? Apache Doris is an open-source, real-time data warehouse. It can collect data from various data sources, including relational databases (MySQL, PostgreSQL, SQL Server, Oracle, etc.), logs, and time series data from IoT devices. It is capable of reporting, ad-hoc analysis, federated queries, and log analysis, so it can be used to support dashboarding, self-service BI, A/B testing, user behavior analysis, and the like. Apache Doris supports both batch import and stream writing.
-
Aug 20, 2023 | 
mdpi.com | Jiafeng Zhang |Guangli Cheng |Jinsong Tang |Haoran Wu
This is an early access version, the complete PDF, HTML, and XML versions will be available soon. Open AccessArticleby, , *, and Institute of Electronic Engineering, Naval University of Engineering, Wuhan 430000, China*Author to whom correspondence should be addressed. J. Mar. Sci. Eng.
-
Aug 9, 2023 | 
hackernoon.com | Jiafeng Zhang
From a data engineer's point of view, financial risk management is a series of data analysis activities on financial data. The financial sector imposes its unique requirements on data engineering. This post explains them with a use case of Apache Doris, and provides reference for what you should take into account when choosing an OLAP engine in a financial scenario. Data Must Be CombinedThe financial data landscape is evolving from standalone to distributed, heterogeneous systems.
-
Jul 31, 2023 | 
hackernoon.com | Jiafeng Zhang
Flink-Doris-Connector 1.4.0 allows users to ingest a whole database (MySQL or Oracle) that contains thousands of tables into Apache Doris, a real-time analytic database, in one step. With built-in Flink CDC, the Connector can directly synchronize the table schema and data from the upstream source to Apache Doris, which means users no long have to write a DataStream program or pre-create mapping tables in Doris.
-
Jul 27, 2023 | 
mdpi.com | Jiafeng Zhang |Jinsong Tang |Zhimin Xie |Guangli Cheng
Abstract:In existing imaging algorithms for wide-beam multiple-receiver synthetic aperture sonar (SAS) systems, the double-square-root (DSR) range history of each receiver is generally converted into the sum of a single-square-root (SSR) range history and an error term using displaced phase center aperture (DPCA) approximation. Therefore, before imaging, each receiver’s error term needs to be individually compensated in the azimuth frequency domain, which is computationally expensive.
-
Jul 25, 2023 | 
hackernoon.com | Jiafeng Zhang
In a data-intensive industry like finance, data comes from numerous entries and goes to numerous exits. Such status quo can easily, and almost inevitably, lead to chaos in data analysis and management. For example, analysts from different business lines define their own financial metrics in data reports. When you pool these countless reports together in your data architecture, you will find that many metrics overlap or even contradict each other in definition.
-
Jul 13, 2023 | 
hackernoon.com | Jiafeng Zhang
This article aims to provide a reference for non-tech companies who are seeking to empower their business with data analytics. You will learn the basics about how to build an efficient and easy-to-use data system, and I will walk you through every aspect of it with a use case of Apache Doris, an MPP-based analytic data warehouse.
-
Jul 3, 2023 | 
hackernoon.com | Jiafeng Zhang
In databases, data update is to add, delete, or modify data. Timely data update is an important part of high quality data services. Technically speaking, there are two types of data updates: you either update a whole row (Row Update) or just update part of the columns (Partial Column Update). Many databases support both of them but in different ways. This post is about one of them, which is simple in execution and efficient in data quality guarantee.
