Monday, October 3, 2011

EXPLORING APPLICATION-LEVEL SEMANTICS FOR DATA COMPRESSION

EXPLORING APPLICATION-LEVEL SEMANTICS FOR

DATA COMPRESSION

ABSTRACT:

Natural phenomena show that many creatures form large social groups and move in regular patterns. However, previous works focus on finding the movement patterns of each single object or all objects. In this paper, we first propose an efficient distributed mining algorithm to jointly identify a group of moving objects and discover their movement patterns in wireless sensor networks. Afterward, we propose a compression algorithm, called 2P2D, which exploits the obtained group movement patterns to reduce the amount of delivered data.

The compression algorithm includes a sequence merge and an entropy reduction phases. In the sequence merge phase, we propose a Merge algorithm to merge and compress the location data of a group of moving objects. In the entropy reduction phase, we formulate a Hit Item Replacement (HIR) problem and propose a Replace algorithm that obtains the optimal solution. Moreover, we devise three replacement rules and derive the maximum compression ratio. The experimental results show that the proposed compression algorithm leverages the group movement patterns to reduce the amount of delivered data effectively and efficiently.

OUR CONTRIBUTIONS ARE THREEFOLD:

· Different from previous works, we formulate a moving object clustering problem that jointly identifies a group of objects and discovers their movement patterns. The application-level semantics are useful for various applications, such as data storage and transmission, task scheduling, and network construction.

EXISTING SYSTEM:

Discovering the group movement patterns is more difficult than finding the patterns of a single object or all objects, because we need to jointly identify a group of objects and discover their aggregated group movement patterns. The constrained resource of WSNs should also be considered in approaching the moving object clustering problem. However, few of existing approaches consider these issues simultaneously. On the one hand, the temporal-and-spatial correlations in the movements of moving objects are modeled as sequential patterns in data mining to discover the frequent movement patterns However, sequential patterns

1) Consider the characteristics of all objects,

2) Lack information about a frequent pattern’s significance regarding individual

trajectories,

3) Carry no time information between consecutive items, which make them

unsuitable for location prediction and similarity comparison.

On the other hand, previous works, such as measure the similarity among these entire trajectory sequences to group moving objects. Since objects may be close together in some types of terrain, such as gorges, and widely distributed in less rugged areas, their group relationships are distinct in some areas and vague in others. Thus, approaches that perform clustering among entire trajectories may not be able to identify the local group relationships. In addition, most of the above works are centralized algorithms which need to collect all data to a server before processing. Thus, unnecessary and redundant data may be delivered, leading to much more power consumption because data transmission needs more power than data processing in Wireless Sensor Networks (WSNs).

PROPOSED SYSTEM:

We have proposed a clustering algorithm to find the group relationships for query and data aggregation efficiency. The differences of and this work are as follows: First, since the clustering algorithm itself is a centralized algorithm, in this work, we further consider systematically combining multiple local clustering results into a consensus to improve the clustering quality and for use in the update-based tracking network. Second, when a delay is tolerant in the tracking application, a new data management approach is required to offer transmission efficiency, which also motivates this study. We thus define the problem of compressing the location data of a group of moving objects as the group data compression problem. We first introduce our distributed mining algorithm to approach the moving object clustering problem and discover group movement patterns. Then, based on the discovered group movement patterns, we propose a novel compression algorithm to tackle the group data compression problem.

Our distributed mining algorithm comprises a Group Movement Pattern Mining (GMPMine) and a Cluster Ensembling (CE) algorithm. It avoids transmitting unnecessary and redundant data by transmitting only the local grouping results to a base station (the sink), instead of all of the moving objects’ location data. Specifically, the GMPMine algorithm discovers the local group movement patterns by using a novel similarity measure, while the CE algorithm combines the local grouping results to remove inconsistency and improve the grouping quality by using the information theory.

Different from previous compression techniques that remove redundancy of data according to the regularity within the data, we devise a novel two-phase and 2D algorithm, called 2P2D, which utilizes the discovered group movement patterns shared by the transmitting node and the receiving node to compress data. In addition to remove redundancy of data according to the correlations within the data of each single object, the 2P2D algorithm further leverages the correlations of multiple objects and their movement patterns to enhance the compressibility.

Architecture Diagrams:

1. Techniques used:


Rounded Rectangle: Get result



2. Process:

Rounded Rectangle: Get Result

Compress data

Jointly identify a group of moving objects and discover their movement patterns in wireless sensor networks

Collect details of moving objects



Module Description:

1) Input data

We have found that many creatures, such as elephants, zebra, whales, and birds, form large social groups when migrating to find food, or for breeding or wintering. These characteristics indicate that the trajectory data of multiple

objects may be correlated for biological applications. Moreover, some research domains, such as the study of animal’s social behavior and wildlife migration, are more concerned with the movement patterns of groups of animals. These details are given as an input data.

2) Apply mining technique

To approach the moving object clustering problem, we propose an efficient distributed mining algorithm to minimize the number of groups such that members in each of the discovered groups are highly related by their movement patterns.

3) Apply compression technique

We propose a novel compression algorithm to compress the location data of a group of moving objects with or without loss of information. We formulate the HIR problem to minimize the entropy of location data and explore the Shannon’s theorem to solve the HIR problem. We also prove that the proposed compression algorithm obtains the optimal solution of the HIR problem efficiently.

4) View result

View the data result that the result contains the mined and compressed data. We exploit the characteristics of group movements to discover the information about groups of moving objects in tracking applications. We propose a distributed

mining algorithm, which consists of a local GMPMine algorithm and a CE algorithm, to discover group movement patterns. With the discovered information, we devise the 2P2D algorithm, which comprises a sequence merge phase and an entropy reduction phase. In the sequence merge phase, we propose the Merge algorithm to merge the location sequences of a group of moving objects with the goal of reducing the overall sequence length. In the entropy reduction phase, we formulate the HIR problem and propose a Replace algorithm to tackle the HIR problem. In addition, we devise and prove three replacement rules, with which the

Replace algorithm obtains the optimal solution of HIR efficiently. Our experimental results show that the proposed compression algorithm effectively reduces the amount of delivered data and enhances compressibility and, by extension, reduces the energy consumption expense for data transmission in WSNs.

HARDWARE & SOFTWARE REQUIREMENTS:

HARDWARE REQUIREMENTS:

· System : Pentium IV 2.4 GHz.

· Hard Disk : 40 GB.

· Floppy Drive : 1.44 Mb.

· Monitor : 15 VGA Colour.

· Mouse : Logitech.

· Ram : 512 MB.

SOFTWARE REQUIREMENTS:

· Operating system : Windows XP Professional.

· Coding Language : ASP .Net,C#

· Database : Sql Server 2005.

EXPLOITING DYNAMIC RESOURCE ALLOCATION FOR EFFICIENT PARALLEL DATA PROCESSING IN THE CLOUD

EXPLOITING DYNAMIC RESOURCE ALLOCATION FOR EFFICIENT PARALLEL DATA PROCESSING IN THE CLOUD

ABSTRACT:

In recent years ad-hoc parallel data processing has emerged to be one of the killer applications for Infrastructure-as-a-Service (IaaS) clouds. Major Cloud computing companies have started to integrate frameworks for parallel data processing in their product portfolio, making it easy for customers to access these services and to deploy their programs. However, the processing frameworks which are currently used have been designed for static, homogeneous cluster setups and disregard the particular nature of a cloud. Consequently, the allocated compute resources may be inadequate for big parts of the submitted job and unnecessarily increase processing time and cost. In this paper we discuss the opportunities and challenges for efficient parallel data processing in clouds and present our research project. It is the first data processing framework to explicitly exploit the dynamic resource allocation offered by today’s IaaS clouds for both, task scheduling and execution. Particular tasks of a processing job can be assigned to different types of virtual machines which are automatically instantiated and terminated during the job execution.


EXISTING SYSTEM:

A growing number of companies have to process huge amounts of data in a cost-efficient manner. Classic representatives for these companies are operators of Internet search engines. The vast amount of data they have to deal with every day has made traditional database solutions prohibitively.

Expensive .Instead, these companies have popularized an architectural paradigm based on a large number of commodity servers. Problems like processing crawled documents or regenerating a web index are split into several independent subtasks, distributed among the available nodes, and computed in parallel.


PROPOSED SYSTEM:

In recent years a variety of systems to facilitate MTC has been developed. Although these systems typically share common goals (e.g. to hide issues of parallelism or fault tolerance), they aim at different fields of application. MapReduce is designed to run data analysis jobs on a large amount of data, which is expected to be stored across a large set of share-nothing commodity servers.

Once a user has fit his program into the required map and reduce pattern, the execution framework takes care of splitting the job into subtasks, distributing and executing them. A single Map Reduce job always consists of a distinct map and reduce program.


ALGORITHMS:

1. Job Scheduling and Execution:

After having received a valid Job Graph from the user, Nephele’s Job Manager transforms it into a so-called Execution Graph. An Execution Graph is Nephele’s primary data structure for scheduling and monitoring the execution of a Nephele job. Unlike the abstract Job Graph, the Execution Graph contains all the concrete information required to schedule and execute the received job on the cloud.

2. Parallelization and Scheduling Strategies:

If constructing an Execution Graph from a user’s submitted Job Graph may leave different degrees of freedom to Nephele. The user provides any job annotation which contains more specific instructions we currently pursue simple default strategy: Each vertex of the Job Graph is transformed into one Execution Vertex. The default channel types are network channels. Each Execution Vertex is by default assigned to its own Execution Instance unless the user’s annotations or other scheduling restrictions (e.g. the usage of in-memory channels) prohibit it.


MODULE DESCRIPTION:

1. NETWORK MODULE:

Server - Client computing or networking is a distributed application architecture that partitions tasks or workloads between service providers (servers) and service requesters, called clients. Often clients and servers operate over a computer network on separate hardware. A server machine is a high-performance host that is running one or more server programs which share its resources with clients. A client also shares any of its resources; Clients therefore initiate communication sessions with servers which await (listen to) incoming requests.

2. LBS SERVICES:

In particular, users are reluctant to use LBSs, since revealing their position may link to their identity. Even though a user may create a fake ID to access the service, her location alone may disclose her actual identity. Linking a position to an individual is possible by various means,

such as publicly available information city maps. When a user u wishes to pose a query, she sends her location to a trusted server, the anonymizer through a secure connection (SSL). The latter obfuscates her location, replacing it with an anonymizing spatial region (ASR) that encloses u. The ASR is then forwarded to the LS. Ignoring where exactly u is, the LS retrieves (and reports to the AZ) a candidate set (CS) that is guaranteed to contain the query results for any possible user location inside the ASR. The AZ receives the CS and reports to u the subset of candidates that corresponds to her original query.

3. SYSTEM MODEL:

The ASR construction at the anonymization process abides by the user’s privacy requirements. Particularly, specified an anonymity degree K by u, the ASR satisfies two properties: (i) it contains u and at least another K * 1 users, and (ii) even if the LS knew the exact locations of all users in the system.

· We propose an edge ordering anonymization approach for users in road networks, which guarantees K-anonymity under the strict reciprocity requirement (described later).

· We identify the crucial concept of border nodes, an important indicator of the CS size and of the query processing cost at the LS.

· We consider various edge orderings, and qualitatively assess their query performance based on border nodes.

· We design efficient query processing mechanisms that exploit existing network database infrastructure, and guarantee CS inclusiveness and minimality. Furthermore, they apply to various network storage schemes.

· We devise batch execution techniques for anonymous queries that significantly reduce the overhead of the LS by computation sharing.

4. SCHEDULED TASK:

Recently, considerable research interest has focused on preventing identity inference in location-based services. Proposing spatial cloaking techniques. In the following, we describe existing techniques for ASR computation (at the AZ) and query processing (at the LS). At the end, we cover alternative location privacy approaches and discuss why they are inappropriate to our problem setting. This offers privacy protection in the sense that the actual user position u cannot be distinguished from others in the ASR, even when malicious LS is equipped/advanced enough to possess all user locations. This spatial K-anonymity model is most widely used in location privacy research/applications, even though alternative models are emerging.

5. QUERY PROCESSING:

Processing is based on implementation of the theorem uses (network-based) search operations as off the shelf building blocks. Thus, the NAP query evaluation methodology is readily deployable on existing systems, and can be easily adapted to different network storage schemes. In this case, the queries are evaluated in a batch. we propose the network-based anonymization and processing (NAP) framework, the first system for K- anonymous query processing in road networks. NAP relies on a global user ordering and bucketization that satisfies reciprocity and guarantees K-anonymity. We identify the ordering characteristics that affect subsequent processing, and qualitatively compare alternatives. Then, we propose query evaluation techniques that exploit these characteristics. In addition to user privacy, NAP achieves low computational and communication costs, and quick responses overall. It is readily deployable, requiring only basic network operations.


HARDWARE & SOFTWARE REQUIREMENTS:

HARDWARE REQUIREMENTS:

· System : Pentium IV 2.4 GHz.

· Hard Disk : 40 GB.

· Floppy Drive : 1.44 Mb.

· Monitor : 15 VGA Colour.

· Mouse : Logitech.

· Ram : 512 MB.

SOFTWARE REQUIREMENTS:

· Operating system : Windows XP Professional.

· Coding Language : ASP .Net,C#

· Database : Sql Server 2005.

EFFICIENT COMPUTATION OF RANGE AGGREGATES AGAINST UNCERTAIN LOCATION BASED QUERIES

EFFICIENT COMPUTATION OF RANGE AGGREGATES AGAINST

UNCERTAIN LOCATION BASED QUERIES

ABSTRACT:

In many applications, including location based services, queries may not be precise. In this paper, we study the problem of efficiently computing range aggregates in a multidimensional space when the query location is uncertain. Specifically, for a query point Q whose location is uncertain and a set S of points in a multi-dimensional space, we want to calculate the aggregate (e.g., count, average and sum) over the subset SI of S, Q has at least probability θ within the distance γ to p. We propose novel, efficient techniques to solve the problem following the filtering-and-verification paradigm. In particular, two novel filtering techniques are proposed to effectively and efficiently remove data points from verification. Our comprehensive experiments based on both real and synthetic data demonstrate the efficiency and scalability of our techniques.


ARCHITECTURE:


ALGORITHM:

FILTERING-AND-VERIFICATION ALGORITHM:

This motivates us to follow the filtering-and-verification paradigm for the uncertain aggregate query computation. Particularly, in the filtering phase, effective and efficient filtering techniques will be applied to prune or validate the points. The algorithm consists of two phases. In the filtering phase for each entry e of RS to be processed, we do not need to further process e if it is pruned or validated by the filter F. We say an entry e is pruned (validated) if the filter can claim Pfall (p, γ) < θ (Pfall (p, γ) ≥ θ) for any point p within embb. The counter cn is increased by |e| if e is validated where |e| denotes the aggregate value of e (i.e., the number of data points in e). Otherwise, the point p associated with e is a candidate point if e corresponds to a data entry and all child entries of e are put into the queue for further processing if e is an intermediate entry. The filtering phase terminates when the queue is empty. In the verification phase candidate points are verified by the integral calculations.


EXISTING SYSTEM:

The existing techniques for processing location based spatial queries regarding certain query points and data points are not applicable or inefficient when uncertain queries are involved.

DISADVANTAGES:

1. Query location is uncertain.

2. In those applications, searching such a location may not only request targets but may also damage other civilian objects details.


PROPOSED SYSTEM:

Our techniques will be presented based on the aggregate count. Nevertheless, they can be immediately extended to cover other aggregates, such as min, max, sum, avg, etc. In this application, the risk of civilian casualties may be measured by the total number n of civilian objects which are within γ distance away from a possible blast point with at least θ probability. It is important to avoid the civilian casualties by estimating the likelihood of damaging civilian objects once the aiming point of a distance (km) is determined.

ADVANTAGES:

1. Avoid damaging civilian objects.

2. Filtering technique is applied.


MODULES:

FILTERING AND VERIFICATION:

When User wants to Search any Civilian Objects like Hotels, Hospitals, Banks, etc., to generate & verify the database then gave exact details. If, he wants to Advanced Search, to give exact range values, apart from that civilian objects details to be displayed with Civilian Objects categorized and calculate the distance details.

QUERY PROCESSING:

Admin Verify the User Requests and check the user requests is contain in our database or not. If the database contains the user requests to calculate the Range (Distance) value, and then send Response to the Appropriate Values, suppose the request doesn’t contain in the database send response is ‘Record Not Found’ that query sent to the particular User.

RESPONSE RESULTS:

User Verify the Admin Responses and the user requests is match with database the query produce with the Correct Results and Download the word file, otherwise not download. If the database contains the user requests to calculate the Range (Distance) value, suppose the request doesn’t contain in the database received response is ‘Record Not Found’ but if matching results are retrieved.

UPLOAD CIVILIAN OBJECTS WITH DISTANCE:

Admin can upload the Civilian Objects details with Correct Location (area), Address, Phone and Upload Civilian Objects Profile (.doc) file. If calculate distance in one location to another location so the admin entry the details for distance and admin can view the User Details also.

SYSTEM REQUIREMENTS:

HARDWARE REQUIREMENTS:

• System : Pentium IV 2.4 GHz.

• Hard Disk : 40 GB.

• Floppy Drive : 1.44 Mb.

• Monitor : 15 VGA Colour.

• Mouse : Logitech.

• Ram : 512 Mb.

SOFTWARE REQUIREMENTS:

• Operating system : Windows XP.

• Coding Language : ASP.Net with C#

• Database : Sql Server 2005.