Evaluation,of,global,navigation,satellite,system,spoofing,efficacy

WANG Yue,SUN Fuping,HAO Jinming,ZHANG Lundong,and WANG Xian

PLA Strategic Support Force Information Engineering University,Henan 450001,China

Abstract:The spoofing capability of Global Navigation Satellite System (GNSS) represents an important confrontational capability for navigation security,and the success of planned missions may depend on the effective evaluation of spoofing capability.However,current evaluation systems face challenges arising from the irrationality of previous weighting methods,inapplicability of the conventional multi-attribute decision-making method and uncertainty existing in evaluation.To solve these difficulties,considering the validity of the obtained results,an evaluation method based on the game aggregated weight model and a joint approach involving the grey relational analysis and technique for order preference by similarity to an ideal solution (GRA-TOPSIS)are firstly proposed to determine the optimal scheme.Static and dynamic evaluation results under different schemes are then obtained via a fuzzy comprehensive assessment and an improved dynamic game method,to prioritize the deceptive efficacy of the equipment accurately and make pointed improvement for its core performance.The use of judging indicators,including Spearman rank correlation coefficient and so on,combined with obtained evaluation results,demonstrates the superiority of the proposed method and the optimal scheme by the horizontal comparison of different methods and vertical comparison of evaluation results.Finally,the results of field measurements and simulation tests show that the proposed method can better overcome the difficulties of existing methods and realize the effective evaluation.

Keywords:Global Navigation Satellite System (GNSS) spoofing,index system for spoofing strategy,game aggregated weight model,grey relational analysis and technique for order preference by similarity to an ideal solution (GRA-TOPSIS) method,dynamic game method.

The increasing complexity of electromagnetic environments is leading to severe challenges in the secure application of the Global Navigation Satellite System (GNSS),rendering research on navigation confrontation to be the most vital part of the study of GNSS safety precautions [1].Compared with the way GNSS is paralyzed by suppressive jamming,deceptive jamming is more effective,harmful,and reliable.As this technology has attracted considerable concern internationally,the development of spoofing equipment has reached the application stage.Therefore,to improve the core performance of these devices and the protective capabilities of system security,effective evaluation for GNSS spoofing efficacy needs to be introduced to help manufacturers select high-performing devices and further strengthen the capabilities of navigation confrontation.The development of related evaluation is not only a manifestation that spoofing technologies are merging to their mature stage,but also a key to accelerate their maturation.

At present,the research on the evaluation of GNSS spoofing efficacy can be divided into two categories:(i) technical evaluation,which is the process of testing and adjusting of performance parameters and their mathematical models of a key technology or equipment;(ii) decision-making evaluation,which is the process of selecting the most suitable technology and maximizing the spoofing benefit of the evaluation system in a specific scenario based on the technical evaluation results.The technical evaluation is the foreshadowing and foundation of decision-making evaluation,and decision-making evaluation is the feedback and sublimation of the technical evaluation.Some research results have been reported on the former.For example,from 2016 to 2019,Tanil et al.[2] used the evaluation model built by the inertial measurement unit (IMU) monitor to detect GNSS spoofing attack activities;Guo et al.[3] and Hu [4] verified and evaluated the performance of spoofing attacks carried out by unmanned aircraft system (UAS) with sensors and receivers.From 2018 to 2020,the indices for evaluating GNSS spoofing efficacy were constructed from navigation signals,positioning results [5],performance of software-hardware to actual utility [6],and then indices modeling [7] and detection by a built testing platform [8] were accomplished.Relatively,few research has been conducted on the latter.Therefore,the research purpose is realizing the decision-making evaluation for GNSS spoofing efficacy,three difficulties are said to hinder its realization.

(i) Because of the multidimensional nature of antagonistic game strategies,the complexity of the electromagnetic environment,diversity of information sources,and differences in expert judgment,the process of evaluating GNSS spoofing efficacy faces inevitable uncertainties,including fuzziness,randomness,and dynamics.In this case,Yu et al.[9] and Yan et al.[10] only pointed out that these uncertainties made it difficult to obtain reasonable evaluation results which could be applied to complex scenarios in reality,without proposing a method to solve it.To determine an optimal solution for the complex conditions,an index system that conforms to reality as much as possible over various spoofing scenarios(schemes) needs to be constructed.

(ii) Navigation confrontation is a multi-attribute decision-making dynamic game problem under conditions of incomplete information.However,conventional multiattribute decision-making methods are of limited use in navigation confrontation.For example,Wang et al.[11]and Chen et al.[12] only revealed defects in the technique for order preference by similarity to an ideal solution (TOPSIS) method when the information was inadequate,including fluctuating large-scale data,difficulty in determining typical distributed rules,and inability to directly reflect changes in trends of index sequence,without solving them.Similarly,Bahan et al.[13] and Ak et al.[14] only utilized conventional multi-attribute decisionmaking methods to obtain an optimal scheme,without further verifying the obtained result by evaluation and decision.In these studies,further static evaluation and dynamic decision were not conducted following scheme ranking,and therefore,they could not corroborate the superiority of the optimal scheme or maximize the spoofing benefit of the evaluated party,thus limiting manufacturers and the security authority from effectively realizing improvement of devices’ performance and secure protective capability.

(iii) The rationality of index weights directly determines the accuracy of the selected optimal scheme and further the overall validity of the evaluation.However,on the one hand,the weighting methods in the multi-scheme evaluation process used in [15,16] were directly weighted to the quantified index obtained under a unified dimension,without considering the difference in the method determination between the subjective indices and objective indices and the inevitable uncertainty during the tradeoff of interval values of indices.On the other hand,previous studies,for example [17] and [18],have focused on the validity of the evaluation methods for selecting an optimal scheme or obtaining the evaluation result,but neglected the validity of the methods for determining a weight.Therefore,the selection of weighting methods can still be considerably improved.

To overcome these difficulties,the following methods are adopted in this research.

(i) An index system fitted to practical confrontational scenarios as close as possible is established,along with characterizing spoofing and anti-spoofing as the relationships of a dynamic game between the attack and defense to establish a hierarchical structure of an antagonistic game corresponding to the spoofing strategy.

(ii) To compensate for the disadvantages of conventional multi-attribute decision-making methods,grey relational analysis (GRA),which is a type of multi-attribute decision-making method for system analysis,is adopted to determine the typical distributed rules and reflect the changing trends in index sequence,and the transitivity of influence relationships between indices when the information is inadequate.It is totally suitable to compensate for the decision-making shortcomings of the TOPSIS.Therefore,a method combining both mathematical models,referred to as the GRA-TOPSIS method,is used in this research to select an optimal scheme under complex conditions.

(iii) To better reduce the uncertainty of performance parameters and maximize the accuracy of the combinational adjustive factors,the game aggregated weight model,which is an improved combinational weighting method to determine the optimal combinational adjustive factors by game theory (GT),combines the interval analytic hierarchy process (IAHP) with a modified entropy weight method through game thoughts to balance subjectivity and objectivity in the process of weights determination,strengthen parameter certainty and further obtain reliable weights for realizing the complete and effective evaluation.

To validate the superiority of the optimal scheme and the proposed method,fully grasp the spoofing efficacy of the evaluated equipment and reduce the uncertainty in the decision-making process,the fuzzy comprehensive assessment (FCA) method,which considers the fuzziness and randomness of influencing factors,and GT,considering the multidimensional nature and dynamic complexity in evaluation process,are respectively used to evaluate the scheme statically and dynamically.However,although GT is commonly used to dynamically evaluate the jamming effects of radar confrontation,for example[19] and [20],existing empirical algorithms and formula models are not completely suitable for the related evaluation of navigation confrontation.Therefore,interval-valued FCA (IFCA) and the weighting model of expert scoring are conjunctively presented to build a benefit matrix as the core module of GT.Finally,the dynamically evaluation and decision is performed by a linear programming algorithm.

According to the classification of GT,navigational confrontation is a dynamic game of incomplete information,under which the choices and actions of the opposing players follow a criterion that the second decision maker grasps only some of the choices and actions of the first decision maker before making their own choices and actions.Under this type of game,the party that first obtains decision information will achieve the initiative,thereby their success rate in performing tasks is signficantly improved.When applying this methodology to navigation confrontation,it is necessary to understand the possible spoofing and anti-spoofing technologies and strategies that can be adopted by both parties before beginning the game.Based on this knowledge,a game matrix of the confrontation can be established to provide guidance for constructing a decision-making evaluation system.

2.1 Game matrix for navigation confrontational scenarios

Under GT,the basic elements of a game are the players,their pure strategy space,and the corresponding benefit matrix.Under the proposed navigation confrontational system,spoofing equipment and GNSS receivers constitute the players of the game and are represented by Atiand Dfj,respectively.Here,eachiandjrepresent a spoofing and an anti-spoofing mode,respectively.The spoofing modes contained by the spoofing equipment constitute their pure strategy space,SAti=[At1,At2,···,At5];the anti-spoofing measures adopted by the receivers constitute their pure strategy space,=[Df1,Df2,···,Df5].Although the goals achieved by the two players and their respective evaluation standards are opposing,they share the same metric of spoofing effect,which is used to form the benefit matrix.

Considering that the spoofing success rates of singleantenna transmission signals and other modes are too low,five spoofing modes and four anti-spoofing measures are selected.For spoofing effectEijobtained when Dfjis selected by the receivers and Atiby the spoofing devices,the game matrix is given as

2.2 Index system for spoofing strategy

Based on the order of signal processed by the target receivers and the principle and characteristics (rapidity,concealment,etc.) of GNSS spoofing technology,six universal indices for each spoofing mode,i.e.,Id1-Id6,and ten specific indices applicable to each spoofing mode,i.e.,Id7-Id16,are selected (as shown in Fig.1).Because realizing the tracking of the synchronized code phase is difficult to the spoofing party,the tracking of the asynchronous code phase is adopted by default,whose specific indices are Id11and Id14.The game situations of navigation confrontation are simplified to Df1,Df2,and Df3(handling for all spoofing modes) and Df4(mainly handling for At1and At4).

Fig.1 Hierarchical structure for spoofing strategies corresponding to confrontational scenarios

To overcome the limitations of conventional multiattribute decision-making methods and irrationality of the weight determination,the GRA-TOPSIS method is combined with the game aggregated weight model to determine the weights of corresponding indices and select an optimal scheme after preprocessing the indices in Fig.1.

3.1 Index pretreatment

To reduce the degree of uncertainty in the evaluation process and ensure that the influencing factors align with actual testing needs,the original interval-valued matrix determined according to the index system for each index,,wheremrepresents the total number of schemes andnis the total number of evaluated indices,can be obtained to further eliminate the influence of the indices with different dimensionalities in evaluation process and facilitate the comparison of the indices.To effectively preprocess allthe extreme-value processing method,which has many advantages including translation independence,difference ratio invariance,and interval stability [21],is applied.The specific expression is given as

wherexijrepresents the preprocessed-valued indicator;is the minimum original value for theith index in thejth confrontational testing scheme,andis the maximum original value corresponding to.

Once the polarization matrixX=(xij)m×nhas been calculated,the decision matrixY=(yij)m×nis obtained via normalization ofXto complete the index preprocessing.

3.2 Interval combinational weighting method based on GT

General combinational weighting method involves a fusion of subjective and objective weighting method to maximize strengths and avoid weaknesses.Considering the randomness in the determination of its linear combinational adjustive factors and the uncertainty in the evaluation process itself,GT is first applied to maximize the accuracy of the combinational adjustive factors.Then,given that the interval theory is more consistent with real scenarios,this GT-based interval combinational weight,i.e.,the game aggregated weight model,is developed as follows.

In terms of subjective empowerment,IAHP considers the randomness of the interval values and the subjectivity of analytic hierarchy process (AHP),which enhances the reliability of index weights relative to AHP alone.Therefore,IAHP is adopted from the subjective aspect as follows.

Step 1The importance of each index is identified using the 1-9 scaling method proposed by Saaty [22].The interval-valued judging matrix is then obtained through consistency detection.Considering the strategy matrix in the antagonistic game as an example,the elements of the judging matrix of the first-level indices are listed in Table 1.

Table 1 Interval-valued judging matrix corresponding to strategy matrix used in antagonistic game

Step 3To further determine the interval benefit matrix,the intervals of the subjective weights are first fixed.Then,a three-element association number algorithm with a good compromise effect is selected according to the method detailed in [23].Finally,the secondary subjective weights of each spoofing modeare obtained as follows:

The first-order weights of each spoofing mode relative to the final evaluation value of spoofing efficacyw∗are obtained:

In terms of objective empowerment,the interval entropy weight method logically considers the fuzziness of the interval values and the objectivity of the entropy weight method,and is thus more widely applicable.The revised interval entropy weight method is adopted from the objective aspect.

Step 1To avoid the characteristic ratio ofyijfrom reaching zero following the preprocessing of the interval values of all indices,the interval entropy weight method is modified by referring to [24] to obtain the entropysij.

By using (8),the secondary objective weights of each spoofing modeare obtained:

To avoid the shortcomings of determining the combinational adjustive factors by some experts in the general combinational weighting methods,GT is applied to obtain a more accuracy combinational adjustive factors and reliable comprehensive weights [25] as follows.

Step 1AsLtypes of the weighting methods has been used to determine index weight in this paper,the set of basic weight vector can be built aswk=[wk1,wk2,···,wkn](k=1,2,···,L).Therefore,any linear combination ofLdifferent vectors could be obtained:

wherewrepresents a possible weight vector in the weight set,andαkis thekth linear combinational adjustive factor.

Step 2When applying the ideas of GT to optimize the advantage ofLtypes of methods,the linear combinational adjustive factorsαkare used to minimize the deviation betweenwand eachwk.

The optimal conditions for the first derivative of (11)are then converted to the following equation:

Step 3With the improved normalized formula forαkobtained by (12),new combinational adjustive factors,,and the final comprehensive weight set are acquired.

Combining the subjective and objective weighting results in (5) and (9) with,the final secondary weights of each index are calculated.

3.3 Principles used to construct GRA-TOPSIS

Step 1Determination of the weighted decision matrixZ.Each column of matrixYis multiplied by the combinational weights of corresponding indexesto obtainzij=yijand thusZ=(zij)m×n.

Step 2Determination of the positive ideal value setofZ.the optimal values corresponding to each index of the respective schemes are screened to form a positive ideal value set used as the reference sequence of the GRA.

Step 3The grey relational coefficientεijof the positive ideal value setcorresponding to each scheme and the grey relational coefficient matrixC=(εij)m×nare given:

Step 4The positive and negative ideal value setsandof the matrixCare determined.

Step 6The relative closenessTiof theith scheme is calculated:

whereTidetermines the generation of the optimal scheme;larger values ofTicorrespond to higher rankings and vice versa.

To further validate the proposed method and the obtained results,and evaluate GNSS spoofing efficacy as effectively as possible,considering the uncertainty and dynamic complexity existed in the evaluation,FCA and improved GT methods are proposed to perform static and dynamic evaluation,to improve the equipment’s core performance on demand and achieve effective evaluation.

4.1 Static evaluation based on FCA

FCA is an analytical method based on fuzzy reasoning,which can solve problems that are difficult to solve via traditional mathematical methods,i.e.,related problems in fuzzy mathematics.The fuzziness and randomness present in the decision-making evaluation could be also reduced via FCA.Therefore,FCA is applied to the static evaluation for the efficacy of GNSS spoofing,whose specific principles and algorithms of this approach could refer to [23].

4.2 Dynamic evaluation based on IFCA-GT

Step 1To distinguish the results more easily by structure,the evaluation set is given five levels,V=(v1,v2,v3,v4,v5),corresponding to excellent,good,medium,average,and poor,respectively.These levels correspond to the interval values of [90,100],[80,90],[70,80],[60,70],and [50,60],respectively.Taking the average interval value corresponding to each level,the following set is obtained:

Step 2Based on our previous research results [27],the triangular with trapezoidal membership function (as shown in Fig.2),which is the most suitable function for evaluating GNSS spoofing efficacy,is applied to obtain the membership.

Fig.2 Simulated charts of triangular with trapezoidal membership function corresponding to noise power intensity

The functions are expressed as follows:

The definitions of all variables in (21)-(25) are summarized in Table 2.

Table 2 Indicator testing values and membership parameters of corresponding spoofing mode

Considering the repeater spoofing mode as an example,the membership of single factors corresponding tov1,v2,v3,v4,andv5are expressed as follows:

Step 3Determination of comprehensive evaluation values.Considering the evaluation set and the membership values of a single factor,the interval fuzzy evaluation value of a single factor is obtained:

Using the interval fuzzy evaluation values and the final secondary weights of a single factor,the interval comprehensive evaluation matrix for the spoofing party is obtained:

Step 4Determination of the spoofing benefit matrix.The comprehensive scoring method developed in [28] is applied by evaluation experts and peer experts to determine the horizontal and vertical weights using a scoring range of 1-10 (low to high).

whereiis a specific spoofing mode,pis the total number of spoofing modes;jis a specific anti-spoofing measure;qis the total number of anti-spoofing measures;bijandbjiare the scores of anti-spoofing measures and spoofing modes.

The five experts produce the following weight sets:J1,J2,J3,J4,andJ5as

Based on the results in [29],different percentages are assigned to the weightsJ1,J2,J3,J4,andJ5.

Step 5Solution of the benefit matrix.In the case of incomplete information,no optimal pure strategy could be achieved,but an optimal mixed strategy is used as follows:

The selection of a mixed strategy based on spoofing devices can get the most adverse outcomeE(X*,Y*) under the most favorable condition,whereas the selection of a mixed strategy based on target receivers can achieve the most favorable outcomeE(X*,Y*) under the most adverse condition.

To obtain the optimal mixed strategy and further dynamic evaluation results,the benefit matrix must be solved first.Compared to other solutions,the linear programming can solve a benefit matrix of any order more conveniently and rapidly.Therefore,the linear algorithm described in detail in [30] is adopted to solve the benefit matrix.

Step 6Determination of the dynamic evaluation results.Through linear programming,the solutions of the benefit matricesELandEUcould be obtained as wherevandωare respectively the expectation values for both sides.Thus,the dynamic evaluation result,i.e.,the range of the minimum spoofing benefit,is as follows:

In real scenarios,the dynamic evaluation results for any spoofing equipment should be within the range in (35).As the spoofing ability of the device in dynamic game increases,the evaluation result grows closer to the maximum of this range.If the result is not within this range,the equipment should fail to persistently deceive because of its poor spoofing ability or because it is monitored by the anti-spoofing party owing to spoofing signals with excessive power.

Based on the steps and processes of the above-mentioned methods as shown in Fig.3,by using testing platforms,detection methods,and distributional schemes,the rankings and evaluation results of the obtained schemes of other classic methods and the proposed method are compared.Setting some related indices as the basis for judgement,the superiority of the optimal scheme and the proposed method is verified to further realize effective decision-making evaluation of GNSS spoofing efficacy.

Fig.3 Steps involved in GNSS spoofing efficacy evaluation

5.1 Source of test data

5.1.1 Construction of the test platforms

To carry out an actual testing analysis to validate the proposed method,testing platforms corresponding to the respective testing methods are constructed as shown in Fig.4.

Fig.4 Testing platforms for evaluating GNSS spoofing efficacy

The hardware-in-the-loop simulation test is performed using a signal simulation testing platform which includes a microwave darkroom and a multisystem satellite navigation signal simulator primarily used to test indices related to navigation signals.The full physical test is usually divided into two types of testing platforms:(i) the full physical static testing platform including a GNSS spoofing signal simulator with related software,a high-frequency oscilloscope,counters,a time interval counter,antennas and a target receiver,is primarily used to test the indicators corresponding to the practical utility under the fixed-point spoofing scenario and the trajectory spoofing scenario,as shown in Fig.5 and Fig.6;(ii) the full physical dynamic testing platform including total stations and a reference station,a timing receiving antenna,a timing receiver,an autonomous robot with related software and a small spoofing device,is primarily used to test the indicators corresponding to positioning results such as spoofing positioning accuracy.

Fig.5 Fixed-point spoofing scenario with project examples by software

Fig.6 Trajectory deception test scenario with corresponding project examples

5.1.2 Detection of the indicators

Six universal indices and one important index corresponding to two types of testing method and three testing platforms described above,are selected for index modeling and detection.The definitions of indices are described as follows.

Definition 1Signal access time: using the full physical static testing platform,the target receiver could receive real navigation signals as well as superimpositions of real signals with the spoofing signal output from the navigation spoofing simulator.The differences between the times at which the simulator started and those at which the signals power generally increased are recorded.

Definition 2Maximum spoofing distance: in the full physical static test,the tested simulator is placed at a specified location,and the target receiver is placed 50 m away from the simulator to determine whether the receiver is deceived near the preset spoofing position.If so,the receiver is moved in intervals of 50 m,and the test is repeated until the receiver loses the lock.

Definition 3Mean absolute deviation of pseudorange,MAD(ρ): in the hardware-in-the-loop simulation test,12 false signals with consistent power are simulated.After normal operation of the receiver for 5 s,the power of each spoofing signal is increased by 1 dB every 10 s then transmitted until the receiver is deceived successfully.Assumingnsuccessful tests,the absolute deviation of pseudo-range is then recordedntimes and averaged to obtain the MAD(ρ).

whereρ0iis the pseudo-range of the preset spoofing position,ρriis the pseudo-range of the measured position after the target receiver has been spoofed,andiis the number of tests.

Definition 4Spoofing position accuracy,root mean square errors (spoofing) (RMSE(s)): in the full physical dynamic test,the target receiver is placed within the deceptive range of the simulator.Twelve spoofing signals with the same power are then output by the simulator.After the receiver has operated normally,the simulator is activated.While the positioning results are stable near the preset spoofing position,the detections are repeatedntimes over the course of one period with seven rounds;the testing results of seven rounds are shown in Fig.7.

Fig.7 Positioning errors XError, YError, and HError in the X-, Y- and H-directions, respectively

The RMSE(s) are calculated as

where (xsj0,ysj0,hsj0) indicates the preset spoofing position of thejth time,whilex,y,andhcorrespond to the abscissa,the ordinate,and the height of the Gaussian rectangular coordinate system,respectively;(xsj,ysj,hsj) is the positioning results of the target receiver after deceived during thejth time.

Definition 5Accuracy of pseudo-range rates: during the full physical static test,a spoofing scenario is carried out in which the satellite is held stationary relative to the user and the carrier frequency value of the counterf0is recorded.The relative speed between the user and the satellite is then set to 1 000 m/s,with the acceleration set to zero,and the carrier frequency valuefiis recorded.The measured values of relative velocityviand the standard deviationσvbetweenviand the preset valuev0are calculated overntests:

Definition 6Timing accuracy of synchronous clock:during the full physical static test,the deviation Δtbetween the 1-pps signal output by the timekeeping unit and the 1-pps signal of an atomic clock is measured by the counter.The simulator is then opened,the antenna is disconnected after 24 h of positioning,and the 1 h information of the time difference output by the counter is recorded.The sums of the front and back deviations are measured at 100 points each,Δtfiand Δt1i,and then averaged.The timing accuracies Δ are recorded as follows:

Definition 7Success rate of spoofing,ω: during the full physical test,the spoofing distance and signal power are specified,and trials are repeatedntimes (each test is conducted for 10 epochs each).If the mean squared error(MSE) between the positioning result of the target receiver and the preset spoofing position does not exceed 5% of the preset position,the deception is considered a success;forgsuccesses,ωis obtained as follows:

5.1.3 Designation of different schemes

Communal testing conditions: based on [27],the initial average powers of the real and spoofing signals are respectively set to -158 dBW and -150 dBW.The ambient noise power,coherent accumulation time,and attenuation are respectively set to -199 dBW,1 ms,and 0 dB.The real signal power varies from -156 dBW to -160 dBW in a step of 1 dB,with the spoofing signal power varing correspondingly from -152 dBW to -148 dBW in a step of 1 dB.For all schemes,each index is tested five times(the time interval for each test is 2 min;the time interval of the indices requires a hot start when testing is 10 s).

Scheme 1The tested equipment is numbered No.AX;other conditions are the same as the communal conditions described above.

Scheme 2The spoofing equipment is numbered No.BX;otherwise,the communal conditions are applied.

Scheme 3The spoofing equipment is numbered No.CX;otherwise,the communal conditions are applied.

5.1.4 Detected results of the indicators

Testing results for the indices (see Fig.1) are obtained for Schemes 1-3 using various testing platforms.Table 2 presents a comparative analysis between interval values of original indices and parameters in the membership function,while Fig.8 shows the pretreatment results for all indices.

Fig.8 Summary of preprocessing results for original indicators

5.2 Determination of optimal scheme

Following the GRA-TOPSIS and based on the results listed in Table 2,the relevancies of Schemes 1-3 are calculated asr1-3=[0.404 4,0.514 4,0.510 9].To verify the superiority of the proposed method,the GRA,TOPSIS,and GRA-TOPSIS based on different weighting methods(the combinational,objective,and subjective weighting methods) are selected to obtain their corresponding correlation degrees and scheme rankings (as shown in Fig.9).In the comparison,the actual effective utilization coefficient (AEUC) of each scheme is taken as the standard quantity,and the results obtained by different methods are compared and analyzed in terms of judging indicators such as the Spearman rank correlation coefficient (significance levelP),range,and coefficient of variation (as shown in Fig.10).

Fig.9 Comparison of correlations obtained by different evaluation methods with standard correlations

Fig.10 Contrastive analysis of results obtained by different evaluation methods based on judging indicators

Larger range values and coefficients of variation correspond to higher-resolution comprehensive evaluation results and larger degrees of dispersion,indicating a greater degree of suitability in terms of distinguishing the operational efficiencies of spoofing strategies under different schemes and the prioritizations of related efficacies for different spoofing equipments.Based on those,Fig.9 and Fig.10 reveal the following.

(i) The rankings of all schemes obtained by the GRA-TOPSIS based on the game aggregated,combinational and objective weighting methods are consistent with the AEUC results,with Spearman rank correlation coefficients relative to AEUC all equal to 1 (P=0);their optimal schemes are Scheme 2,while the worst schemes obtained by the GRA and TOPSIS methods are Scheme 2.

(ii) The range and coefficient of variation of the results obtained by the GRA-TOPSIS based on the game aggregated weight model are larger than those obtained by the GRA-TOPSIS based on the combinational and objective weighting methods.

These results verify the superiority of the proposed method directly and the superiority of the obtained optimal scheme indirectly,by transverse comparative analysis.

5.3 Comparison of static evaluation results

The FCA is applied to obtain the second-level evaluation matrixf1-3(At) of each spoofing mode.By combining this matrix with first-level weightsw∗in (6),the static evaluation results of Schemes 1-3 are weighted as

Formula (41) reveals that the ranking of the static evaluation results for Schemes 1-3 is completely consistent with the ranking of Schemes 1-3 obtained by the proposed method,i.e.,the top-ranked scheme of both is Scheme 2,which indirectly validates the superiority of the proposed method and reveals the advantages of adjusting the performance of spoofing equipment and feeding related performance improved details back to the makers by the proposed method.

5.4 Comparison of dynamic evaluation results

Using the dynamic evaluation method based on the testing platforms,when the target receiver is spoofed,the benefit matrices of Schemes 1-3 relative to spoofing modesE1,E2,andE3are presented as follows:

Formula (42) reveals the minimum spoofing benefit of Schemes 1-3,i.e.,dynamic evaluation results,areE1=44.459 2,E2=46.199 8 andE3=45.739 6.The largest result is the result of Scheme 2,which validates the superiority of the proposed method indirectly and the superiority of the optimal scheme directly.Finally,the degrees of distance are compared between dynamic evaluation results of Schemes 1-3 and the range of the spoofing benefit [EL,EU] obtained by (35),as shown in Fig.11.

Fig.11 Comparison of the distance degrees of the dynamic evaluation results and interval values

The further away the dynamic evaluation result relative to the minimumELis,the closer relative to the maximumEU,which further indicates that when this dynamic evaluation result is larger,its corresponding scheme is better,and the spoofing device included in this scheme has strong spoofing capability in real confrontational scenarios.Fig.11 shows the following:

(i) Scheme 2 meets the above conditions,which verifies the superiority of the proposed method indirectly and the superiority of the optimal scheme directly,and improves the success rate of spoofing for the missions.

(ii) In actual confrontational scenarios,the performance of the equipment contained in Scheme 2 can be used as a standard to compare it with other devices for various indices and help these devices perfect related property.

Combined with the analysis of the actual cases,the optimal scheme is selected by the GRA-TOPSIS based on the game aggregated weight model;the proposed method and the optimal result are directly and indirectly verified by horizontal and vertical comparative analysis using different methods and evaluation results.The conclusions are drawn as follows.

(i) The optimal schemes obtained by GRA-TOPSIS based on any kind of weighting methods are Scheme 2;the range and coefficient of variation of the results obtained by the GRA-TOPSIS based on the game aggregated weight model are larger than those obtained by the GRA-TOPSIS based on the other weighting methods.To summarize up,the proposed method exhibites superior performance compared to other related methods,and validates the superiority of the obtained optimal scheme.Therefore,the spoofing equipment (XB) included in the optimal scheme,which has the strongest deceptive ability in confrontational scenarios,should be prioritized for use in actual complex conditions,and as a standard for perfecting other equipment and improving overall spoofing ability.

(ii) Compared with conventional multi-attribute decision-making methods and weighting methods,the proposed method reduces the uncertainty in the decisionmaking evaluation,proposes a reasonable weighting method applicable to evaluation of GNSS spoofing efficacy,solves the inapplicability of traditional multiattribute decision in incomplete information and dynamic game in navigation field,verifies the superiority of the proposed method and obtains result directly and indirectly based on the horizontal comparison using different methods and vertical comparison using evaluation results,and further realizes the decision-making evaluation by dynamically adjusting the spoofing strategies related to real scenarios.However,there are some deficiencies.

(iii) The accuracy and validity of the benefit matrix of the spoofing party have not been verified and the software system for the evaluation of GNSS spoofing efficacy is not complete.Therefore,in the later stage,a feedback system connecting the evaluation results to the benefit matrix should be constructed to verify and adjust the benefit matrix by evaluating the quality of the results,and a software system for evaluating and deciding should be developed based on testing values of the indices and improved mathematical models in operational research,to improve the efficiency of the decision-making evaluation and further maximize the spoofing benefit of the spoofing party.

推荐访问:Navigation Global Evaluation