Introdution
AddrMiner: a systematic and comprehensive global active IPv6 address probing system.AddrMiner divides the IPv6 address space regions into three kinds according to the number of seed addresses to discover active IPv6 addresses from scratch, from few to many. AddrMiner would like to further open the door of IPv6 measurement studies by publicly releasing AddrMiner and sharing our data.
The System Architecture of AddrMiner
Our latest research incorporates real-world operational conditions, simplifies the system architecture, and divides it into two scenarios (seeded address scenarios and seedless address scenarios). Our latest research will be made public soon.
Our IPv6 Hitlist collection process
We continuously monitor globally active IPv6 addresses. The growth curve of our IPv6 hitlist is depicted in the graph below. The activity of IPv6 addresses exhibits a temporal nature, meaning that an address may be active at one moment and not be used in the next. Our project focuses on identifying consistently active IPv6 addresses, so we monitor the activity of collected active IPv6 addresses daily. To maintain the availability of our IPv6 hitlist (with high activity), we regularly remove long-term inactive IPv6 addresses. For instance, on November 1, 2022, we cleaned our historical data collection by removing over one billion long-term inactive IPv6 addresses. We provide an IPv6 Hitlist service to provide active IPv6 addresses to interested researchers.Openly Accessible Service
- Active IPv6 address list(IPv6 hitlist):
Referencing the Hitlist Service
If you are using data from Our IPv6 Hitlist in your publication, please refer to it with the following references(Addrminer [bib], DET [bib], and SongTowards [bib]):
@inproceedings{Addrminer,
title={$\{$AddrMiner$\}$: A Comprehensive Global Active $\{$IPv6$\}$ Address Discovery System},
author={Song, Guanglei and Yang, Jiahai and He, Lin and Wang, Zhiliang and Li, Guo and Duan, Chenxin and Liu, Yaozhong and Sun, Zhongxiang},
booktitle={2022 USENIX Annual Technical Conference (USENIX ATC 22)},
pages={309--326},
year={2022}
}
@article{DET,
title={Det: Enabling efficient probing of ipv6 active addresses},
author={Song, Guanglei and Yang, Jiahai and Wang, Zhiliang and He, Lin and Lin, Jinlei and Pan, Long and Duan, Chenxin and Quan, Xiaowen},
journal={IEEE/ACM Transactions on Networking},
volume={30},
number={4},
pages={1629--1643},
year={2022},
publisher={IEEE}
}
@inproceedings{SongTowards,
title={Towards the construction of global IPv6 hitlist and efficient probing of IPv6 address space},
author={Song, Guanglei and He, Lin and Wang, Zhiliang and Yang, Jiahai and Jin, Tao and Liu, Jieling and Li, Guo},
booktitle={2020 IEEE/ACM 28th International Symposium on Quality of Service (IWQoS)},
pages={1--10},
year={2020},
organization={IEEE}
}
Software and Tools
We publish the following software and tools for use by the scientific community:AddrMiner-v2.0
AddrMiner-v2.0: is an improved global active IPv6 address detection system based on AddrMiner-v1.0. The specific improvements encompass two aspects: on the one hand, it simplifies the address detection architecture by dividing address detection into two detection scenarios, making it more practical for deployment. On the other hand, it introduces a multi-level association policy (MLAP) to enhance further the coverage and resource utilization of seedless regional detection.Source: github.com/AddrMiner/AddrMiner-v2.0
AddrMiner-v1.0
AddrMiner-v1.0: is a systematic, comprehensive, and efficient active IPv6 address detection system. It divides address detection into three scenarios: seedless address scenario, few seeded address scenario, and sufficient seeded address scenario.Source: github.com/AddrMiner/AddrMiner-v1.0
DET
DET is an abbreviation for detective. It is an efficient IPv6 address discovery method that combines density, information entropy, and spatial tree.Source: github.com/sixiangdeweicao/DET
FAPD
FAPD: is a fingerprint-based method to detect aliased prefixes.Source: github.com/sixiangdeweicao/Aliased_prefixs_TBT
Random-Bytes
Random-Bytes: is a method to generate target addresses under each BGP prefix based on a common pattern of observation.Source: github.com/sixiangdeweicao/Random-Bytes
6Gen-optimization
Gen-optimization: An optimized version of 6Gen, which discovers seed address high-density areas in linear time and detects active IPv6 addresses in high-density regions.Source: github.com/sixiangdeweicao/6Gen-optimization
AddrMiner: A Comprehensive Global Active IPv6 Address Discovery System
Abstract. Fast Internet-wide scanning is essential for network situational awareness and asset evaluation. However, the vast IPv6 address space makes brute-force scanning infeasible. Although state-of-the-art techniques have made effective attempts, these methods do not work in seedless regions, while the detection efficiency is low in regions with seeds. Moreover, the constructed hitlist with low coverage cannot truly represent the active IPv6 address landscape of the Internet.This paper introduces AddrMiner, a global active IPv6 address probing system, making IPv6 active address probing systematic, comprehensive, and economical. We divide the IPv6 address space regions into three kinds according to the number of seed addresses and propose a probing algorithm for each of them. For the regions with no seeds, we propose AddrMiner-N, leveraging an organization association strategy to mine active addresses. It finds active addresses covering 86.4K BGP prefixes, accounting for 81.6% of the probed BGP prefixes. For the regions with few seeds, we propose AddrMiner-F, utilizing a similarity matching strategy to probe active addresses further. The hit rate of active address probing is improved by 70%-150% compared to existing algorithms. For the regions with sufficient seeds, we propose AddrMiner-S to generate target addresses based on reinforcement learning dynamically. It nearly doubles the hit rate compared to the state-of-the-art algorithms. Finally, we deploy AddrMiner and discover 2.1 billion active IPv6 addresses, including 1.7 billion de-aliased active addresses and 0.4 billion aliased addresses, through continuous probing for 13 months. We would like to further open the door of IPv6 measurement studies by publicly releasing AddrMiner and sharing our data.
Paper. Read the final version of our paper here: [PDF]
Authors. Guanglei Song , Jiahai Yang, Lin He, Zhiliang Wang, Guo Li, Chenxin Duan, Yaozhong Liu, Zhongxiang Sun
Published in: Usenix ATC 2022
Det: Enabling efficient probing of ipv6 active addresses
Abstract. Fast IPv4 scanning significantly improves network measurement and security research. Nevertheless, it is infeasible to perform brute-force scanning of the IPv6 address space. Alternatively, one can find active IPv6 addresses through scanning the candidate addresses generated by state-of-the-art algorithms. However, the probing efficiency of such algorithms is often very low. In this paper, our objective is to improve the probing efficiency of IPv6 addresses. We first perform a longitudinal active measurement study and build a high-quality dataset, hitlist, including more than 1.95B IPv6 addresses distributed in 58.2K BGP prefixes and collected over 17 months period. Different from the previous works, we probe the announced BGP prefixes using a pattern-based algorithm. This results in a dataset without uneven address distribution and low active rates. Further, we propose an efficient address generation algorithm, DET, which builds a density space tree to learn high-density address regions of the seed addresses with linear time complexity and improves the active addresses' probing efficiency. We then compare our algorithm DET against state-of-the-art algorithms on the public hitlist and our hitlist by scanning 50M addresses. Our analysis shows that DET increases the de-aliased active address ratio and active address (including aliased addresses) ratio by 10%, and 14%, respectively. Furthermore, we develop a fingerprint-based method to detect aliased prefixes. The proposed method for the first time directly verifies whether the prefix is aliased or not. Our method finds that 10.64% of the public aliased prefixes are false positive.Paper. Read the final version of our paper here: [PDF]
Authors. Guanglei Song , Jiahai Yang, Zhiliang Wang, Lin He, Jinlei Lin, Long Pan, Chenxin Duan, Xiaowen Quan.
Published in: IEEE/ACM Transactions on Networking ( Volume: 30, Issue: 4, August 2022)
Towards the construction of global IPv6 hitlist and efficient probing of IPv6 address space
Abstract. Fast IPv4 scanning has made sufficient progress in network measurement and security research. However, it is infeasible to perform brute-force scanning of the IPv6 address space. We can find active IPv6 addresses through scanning candidate addresses generated by the state-of-the-art algorithms, whose probing efficiency of active IPv6 addresses, however, is still very low. In this paper, we aim to improve the probing efficiency of IPv6 addresses in two ways. Firstly, we perform a longitudinal active measurement study over four months, building a high-quality dataset called hitlist with more than 1.3 billion IPv6 addresses distributed in 45.2k BGP prefixes. Different from previous work, we probe the announced BGP prefixes using a pattern-based algorithm, which makes our dataset overcome the problems of uneven address distribution and low active rate. Secondly, we propose an efficient address generation algorithm DET, which builds a density space tree to learn high-density address regions of the seed addresses in linear time and improves the probing efficiency of active addresses. On the public hitlist and our hitlist, we compare our algorithm DET against state-of-the-art algorithms and find that DET increases the de-aliased active address ratio by 10%, and active address (including aliased addresses) ratio by 14%, by scanning 50 million addresses.Paper. Read the final version of our paper here: [PDF]
Authors. Guanglei Song , Lin He, Zhiliang Wang, Jiahai Yang, Tao Jin, Jieling Liu, Guo Li.
Published in: 2020 IEEE/ACM 28th International Symposium on Quality of Service (IWQoS)
License
Our work has received multiple patent authorizations. We provide free support exclusively for academic research and do not support any commercial activities. Anyone intending to apply our technology and data for commercial purposes should contact us in advance and obtain authorization before use. (sixiangdeweicao@gmail.com)Contact
Guanglei Song: sixiangdeweicao@gmail.comIn order to support IPv6 network-related research, we provide more data about hitlist(active IPv6 addresses) and address fingerprint information. If you want more data, you can send a request to sixiangdeweicao@gmail.com. The request should include the work department, the purpose of data usage, and the data content obtained.
Partners
