Mẹo Which of the following is not an important trend in data communications and networking
Kinh Nghiệm Hướng dẫn Which of the following is not an important trend in data communications and networking 2022
Lê Minh Châu đang tìm kiếm từ khóa Which of the following is not an important trend in data communications and networking được Cập Nhật vào lúc : 2022-10-06 17:38:05 . Với phương châm chia sẻ Kinh Nghiệm Hướng dẫn trong nội dung bài viết một cách Chi Tiết Mới Nhất. Nếu sau khi đọc tài liệu vẫn ko hiểu thì hoàn toàn có thể lại phản hồi ở cuối bài để Mình lý giải và hướng dẫn lại nha.What is big data?
Big data is a combination of structured, semistructured and unstructured data collected by organizations that can be mined for information and used in machine learning projects, predictive modeling and other advanced analytics applications.
Nội dung chính- What is big data?Why is big data important?What are examples of big data?Breaking down the V's of big dataMore characteristics of big dataHow is big data stored and processed?How big data analytics worksBig data management technologiesBig data challengesKeys to an effective big data strategyBig data collection practices and regulationsThe human side of big data management and analytics
Systems that process and store big data have become a common component of data management architectures in organizations, combined with tools that support big data analytics uses. Big data is often characterized by the three V's:
- the large volume of data in many environments;the wide variety of data types frequently stored in big data systems; andthe velocity which much of the data is generated, collected and processed.
These characteristics were first identified in 2001 by Doug Laney, then an analyst consulting firm Meta Group Inc.; Gartner further popularized them after it acquired Meta Group in 2005. More recently, several other V's have been added to different descriptions of big data, including veracity, value and variability.
Although big data doesn't equate to any specific volume of data, big data deployments often involve terabytes, petabytes and even exabytes of data created and collected over time.
Why is big data important?
Companies use big data in their systems to improve operations, provide better customer service, create personalized marketing campaigns and take other actions that, ultimately, can increase revenue and profits. Businesses that use it effectively hold a potential competitive advantage over those that don't because they're able to make faster and more informed business decisions.
For example, big data provides valuable insights into customers that companies can use to refine their marketing, advertising and promotions in order to increase customer engagement and conversion rates. Both historical and real-time data can be analyzed to assess the evolving preferences of consumers or corporate buyers, enabling businesses to become more responsive to customer wants and needs.
Big data is also used by medical researchers to identify disease signs and risk factors and by doctors to help diagnose illnesses and medical conditions in patients. In addition, a combination of data from electronic health records, social truyền thông sites, the web and other sources gives healthcare organizations and government agencies up-to-date information on infectious disease threats or outbreaks.
Here are some more examples of how big data is used by organizations:
- In the energy industry, big data helps oil and gas companies identify
potential drilling locations and monitor pipeline operations; likewise, utilities use it to track electrical grids.Financial services firms use big data systems for risk management and real-time analysis of market data.Manufacturers and transportation companies rely on big data to manage their supply chains and optimize delivery routes.
Other government uses include emergency response, crime prevention and smart city initiatives.
These are some of the business benefits organizations can get by using big
data. What are examples of big data?
Big data comes from myriad sources -- some examples are transaction processing systems, customer databases, documents, emails, medical records, internet clickstream logs, mobile apps and social networks. It also includes machine-generated data, such as network and server log files and data from sensors on manufacturing machines, industrial equipment and internet of things devices.
In addition to data from internal systems, big data environments often incorporate external data on consumers, financial markets, weather and traffic conditions, geographic information, scientific research and more. Images, videos and audio files are forms of big data, too, and many big data applications involve streaming data that is processed and collected on a continual basis.
Breaking down the V's of big data
Volume is the most commonly cited characteristic of big data. A big data environment doesn't have to contain a large amount of data, but most do because of the nature of the data being collected and stored in them. Clickstreams, system logs and stream processing systems are among the sources that typically produce massive volumes of data on an ongoing basis.
Big data also encompasses a wide variety of data types, including the following:
- structured data, such as transactions and financial records;unstructured data, such as text, documents and multimedia files; andsemistructured data, such as web server logs and streaming data from sensors.
Various data types may need to be stored and managed together in big data systems. In addition, big data applications often include multiple data sets that may not be integrated upfront. For example, a big data analytics project may attempt to forecast sales of a product by correlating data on past sales, returns, online reviews and customer service calls.
Velocity refers to the speed which data is generated and must be processed and analyzed. In many cases, sets of big data are updated on a real- or near-real-time basis, instead of the daily, weekly or monthly updates made in many traditional data warehouses. Managing data velocity is also important as big data analysis further expands into machine learning and artificial intelligence (AI), where analytical processes automatically find patterns in data and use them to generate insights.
[embed]https://www.youtube.com/watch?v=jH44SfUNpWw[/embed]
More characteristics of big data
Looking beyond the original three V's, here are details on some of the other ones that are now often associated with big data:
- Veracity refers to the degree of accuracy in data sets and how trustworthy they are. Raw data collected from various sources can cause data quality issues that may be difficult to pinpoint. If they aren't fixed through
data cleansing processes, bad data leads to analysis errors that can undermine the value of business analytics initiatives. Data management and analytics teams also need to ensure that they have enough accurate data available to produce valid results.Some data scientists and
consultants also add value to the list of big data's characteristics. Not all the data that's collected has real business value or benefits. As a result, organizations need to confirm that data relates to relevant business issues before it's used in big data analytics projects.Variability also often applies to sets of big data, which may have multiple meanings or be formatted differently in separate data sources -- factors that further complicate
big data management and analytics.
Some people ascribe even more V's to big data; various lists have been created with between seven and 10.
The characteristics of big data are
commonly described by using words that begin with 'v,' including these six. How is big data stored and processed?
Big data is often stored in a data lake. While data warehouses are commonly built on relational databases and contain structured data only, data lakes can support various data types and typically are based on Hadoop clusters, cloud object storage services, NoSQL databases or other big data platforms.
Many big data environments combine multiple systems in a distributed architecture; for example, a central data lake might be integrated with other platforms, including relational databases or a data warehouse. The data in big data systems may be left in its raw form and then filtered and organized as needed for particular analytics uses. In other cases, it's preprocessed using data mining tools and data preparation software so it's ready for applications that are run regularly.
Big data processing places heavy demands on the underlying compute infrastructure. The required computing power often is provided by clustered systems that distribute processing workloads across hundreds or thousands of commodity servers, using technologies like Hadoop and the Spark processing engine.
Getting that kind of processing capacity in a cost-effective way is a challenge. As a result, the cloud is a popular location for big data systems. Organizations can deploy their own cloud-based systems or use managed big-data-as-a-service offerings from cloud providers. Cloud users can scale up the required number of servers just long enough to complete big data analytics projects. The business only pays for the storage and compute time it uses, and the cloud instances can be turned off until they're needed again.
How big data analytics works
To get valid and relevant results from big data analytics applications, data scientists and other data analysts must have a detailed understanding of the available data and a sense of what they're looking for in it. That makes data preparation, which includes profiling, cleansing, validation and transformation of data sets, a crucial first step in the analytics process.
Once the data has been gathered and prepared for analysis, various data science and advanced analytics disciplines can be applied to run different applications, using tools that provide big data analytics features and capabilities. Those disciplines include machine learning and its deep learning offshoot, predictive modeling, data mining, statistical analysis, streaming analytics, text mining and more.
Using customer data as an example, the different branches of analytics that can be done with sets of big data include the following:
- Comparative analysis. This examines customer behavior metrics and real-time customer engagement in order to compare a company's products, services and branding with those of its competitors.Social truyền thông listening. This analyzes what people are saying on
social truyền thông about a business or product, which can help identify potential problems and target audiences for marketing campaigns.Marketing analytics. This provides information that can be used to improve marketing campaigns and promotional offers for products, services and business initiatives.Sentiment analysis. All of the data that's gathered on customers can be analyzed to reveal how they feel about a company or brand,
customer satisfaction levels, potential issues and how customer service could be improved.
Big data management technologies
Hadoop, an open source distributed processing framework released in 2006, initially was the center of most big data architectures. The development of Spark and other processing engines pushed MapReduce, the engine built into Hadoop, more to the side. The result is an ecosystem of big data technologies that can be used for different applications but often are deployed together.
Big data platforms and managed services offered by IT vendors combine many of those technologies in a single package, primarily for use in the cloud. Currently, that includes these offerings, listed alphabetically:
- Amazon EMR (formerly Elastic MapReduce)Cloudera Data PlatformGoogle Cloud DataprocHPE Ezmeral Data Fabric (formerly MapR Data Platform)Microsoft Azure HDInsight
For organizations that want to deploy big data systems themselves, either on premises or in the cloud, the technologies that are available to them in addition to Hadoop and Spark include the following categories of tools:
- storage repositories, such as the Hadoop Distributed File System (HDFS) and cloud object storage services that include Amazon
Simple Storage Service (S3), Google Cloud Storage and Azure Blob Storage;cluster management frameworks, like Kubernetes, Mesos and YARN, Hadoop's built-in resource manager and job scheduler, which stands for Yet Another Resource
Negotiator but is commonly known by the acronym alone;stream processing engines, such as Flink, Hudi, Kafka, Samza, Storm and the Spark Streaming and Structured Streaming modules built into Spark;NoSQL databases that include Cassandra, Couchbase, CouchDB, HBase, MarkLogic Data Hub, MongoDB, Neo4j, Redis and various other technologies;data lake and data warehouse platforms, among them Amazon Redshift, Delta Lake, Google BigQuery, Kylin and Snowflake; andSQL
query engines, like Drill, Hive, Impala, Presto and Trino.
Big data challenges
In connection with the processing capacity issues, designing a big data architecture is a common challenge for users. Big data systems must be tailored to an organization's particular needs, a DIY undertaking that requires IT and data management teams to piece together a customized set of technologies and tools. Deploying and managing big data systems also require new skills compared to the ones that database administrators and developers focused on relational software typically possess.
Both of those issues can be eased by using a managed cloud service, but IT managers need to keep a close eye on cloud usage to make sure costs don't get out of hand. Also, migrating on-premises data sets and processing workloads to the cloud is often a complex process.
Other challenges in managing big data systems include making the data accessible to data scientists and analysts, especially in distributed environments that include a mix of different platforms and data stores. To help analysts find relevant data, data management and analytics teams are increasingly building data catalogs that incorporate metadata management and data lineage functions. The process of integrating sets of big data is often also complicated, particularly when data variety and velocity are factors.
Keys to an effective big data strategy
In an organization, developing a big data strategy requires an understanding of business goals and the data that's currently available to use, plus an assessment of the need for additional data to help meet the objectives. The next steps to take include the following:
- prioritizing planned use cases and applications;identifying new systems and tools that are needed;creating a deployment roadmap; andevaluating internal skills to see if retraining or hiring are required.
To ensure that sets of big data are clean, consistent and used properly, a data governance program and associated data quality management processes also must be priorities. Other best practices for managing and analyzing big data include focusing on business needs for information over the available technologies and using data visualization to aid in data discovery and analysis.
Big data collection practices and regulations
As the collection and use of big data have increased, so has the potential for data misuse. A public outcry about data breaches and other personal privacy violations led the European Union to approve the General Data Protection Regulation (GDPR), a data privacy law that took effect in May 2022. GDPR limits the types of data that organizations can collect and requires opt-in consent from individuals or compliance with other specified reasons for collecting personal data. It also includes a right-to-be-forgotten provision, which lets EU residents ask companies to delete their data.
While there aren't similar federal laws in the U.S., the California Consumer Privacy Act (CCPA) aims to give California residents more control over the collection and use of their personal information by companies that do business in the state. CCPA was signed into law in 2022 and took effect on Jan. 1, 2022.
To ensure that they comply with such laws, businesses need to carefully manage the process of collecting big data. Controls must be put in place to identify regulated data and prevent unauthorized employees from accessing it.
The human side of big data management and analytics
Ultimately, the business value and benefits of big data initiatives depend on the workers tasked with managing and analyzing the data. Some big data tools enable less technical users to run predictive analytics applications or help businesses deploy a suitable infrastructure for big data projects, while minimizing the need for hardware and distributed software know-how.
Big data can be contrasted with small data, a term that's sometimes used to describe data sets that can be easily used for self-service BI and analytics. A commonly quoted axiom is, "Big data is for machines; small data is for people."
Tải thêm tài liệu liên quan đến nội dung bài viết Which of the following is not an important trend in data communications and networking
Post a Comment