A multiplication overflow problem

Technology
Word count: 2.2k   |   Reading time≈ 11 min

Preface

我司产品从内核ct抄了一份ip解析的代码在应用层使用，在客户跑业务的时候coredump掉了，从堆栈分析看是做乘法过程中整数溢出abort了，在这里简单记录下。

in4_pton

问题函数为in4_pton，用于将字符串ip转换成二进制ip，定义如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
/**
 * in4_pton - convert an IPv4 address from literal to binary representation
 * @src: the start of the IPv4 address string
 * @srclen: the length of the string, -1 means strlen(src)
 * @dst: the binary (u8[4] array) representation of the IPv4 address
 * @delim: the delimiter of the IPv4 address in @src, -1 means no delimiter
 * @end: A pointer to the end of the parsed string will be placed here
 *
 * Return one on success, return zero when any error occurs
 * and @end will point to the end of the parsed string.
 *
 */
int in4_pton(const char *src, int srclen,
	     u8 *dst,
	     int delim, const char **end)
{
	const char *s;
	u8 *d;
	u8 dbuf[4];
	int ret = 0;
	int i;
	int w = 0;

	if (srclen < 0)
		srclen = strlen(src);
	s = src;
	d = dbuf;
	i = 0;
	while (1) {
		int c;
		c = xdigit2bin(srclen > 0 ? *s : '\0', delim);
		if (!(c & (IN6PTON_DIGIT | IN6PTON_DOT | IN6PTON_DELIM | IN6PTON_COLON_MASK))) {
			goto out;
		}
		if (c & (IN6PTON_DOT | IN6PTON_DELIM | IN6PTON_COLON_MASK)) {
			if (w == 0)
				goto out;
			*d++ = w & 0xff;
			w = 0;
			i++;
			if (c & (IN6PTON_DELIM | IN6PTON_COLON_MASK)) {
				if (i != 4)
					goto out;
				break;
			}
			goto cont;
		}
		w = (w * 10) + c; /* Coredump here. */
		if ((w & 0xffff) > 255) {
			goto out;
		}
cont:
		if (i >= 4)
			goto out;
		s++;
		srclen--;
	}
	ret = 1;
	memcpy(dst, dbuf, sizeof(dbuf));
out:
	if (end)
		*end = s;
	return ret;
}
EXPORT_SYMBOL(in4_pton);

导致coredump的代码为：

1
w = (w * 10) + c;

coredump堆栈为：

1
2
3
4
5
#0  0x00007f9563a35428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007f9563a3702a in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x000056378c35c332 in __mulvsi3 ()
#3  0x000056378bdcec16 in in4_pton (delim=-1, end=<synthetic pointer>, dst=0x7f95355e9a44 "", srclen=566, 
    src=0x50005e1169ba <error: Cannot access memory at address 0x50005e1169ba>)

这里src是mbuf的L4数据起始地址，由于环境上没有打开大页内存的coredump，导致src的内容无法获取，提高了一点分析难度，不过影响也不是很大。

接着分析下整体函数流程，从src的起始地址s开始读取每个字符并作相应转换，保存到c中，然后累加到w中去，等于逐位读取十进制整数：

1
2
3
4
5
6
7
8
9
......
		int c;
		c = xdigit2bin(srclen > 0 ? *s : '\0', delim);
......
		w = (w * 10) + c; /* Coredump here. */
		if ((w & 0xffff) > 255) {
			goto out;
		}
......

很明显，这一段是为了将字符形式的点分十进制的IP的数字部分转换成十进制数字并保存，这里通过累计值是否大于255判断数值是否溢出/格式是否正确。

这里要注意的一点是，从xdigit2bin定义中可以发现，c的高16bit代表了c字符的类型，低16bit代表了对应的数值：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
#define IN6PTON_XDIGIT		0x00010000
#define IN6PTON_DIGIT		0x00020000
#define IN6PTON_COLON_MASK	0x00700000
#define IN6PTON_COLON_1		0x00100000	/* single : requested */
#define IN6PTON_COLON_2		0x00200000	/* second : requested */
#define IN6PTON_COLON_1_2	0x00400000	/* :: requested */
#define IN6PTON_DOT		0x00800000	/* . */
#define IN6PTON_DELIM		0x10000000
#define IN6PTON_NULL		0x20000000	/* first/tail */
#define IN6PTON_UNKNOWN		0x40000000

static inline int xdigit2bin(char c, int delim)
{
	int val;

	if (c == delim || c == '\0')
		return IN6PTON_DELIM;
	if (c == ':')
		return IN6PTON_COLON_MASK;
	if (c == '.')
		return IN6PTON_DOT;

	val = hex_to_bin(c);
	if (val >= 0)
		return val | IN6PTON_XDIGIT | (val < 10 ? IN6PTON_DIGIT : 0);

	if (delim == -1)
		return IN6PTON_DELIM;
	return IN6PTON_UNKNOWN;
}

也正是这一点导致了最终的溢出：如果c只表示数值的话，通过和255比较可以确保不溢出，但每个数字的高16bit都有置位，当起始字符串的以若干0开头时，高16bit一直被累加，而低16bit的值一直是0，始终小于255，最终整体超过了int的上界，结果溢出。

How about kernel?

单单看溢出原因好像已经很明确了，但这段代码是从内核抄出来的，为什么内核正常运行安稳无恙呢？

所以在内核上构造一下对应场景验证，函数调用顺序为：nf_nat_sip->ct_sip_parse_request->skp_epaddr_len->epaddr_len->sip_parse_addr->in4_pton。

顶层调用入口nf_nat_sip位于内核模块nf_nat_sip，在模块初始化的时候被注册到netfilter的NAT模块中：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
static const struct nf_nat_sip_hooks sip_hooks = {
	.msg		= nf_nat_sip,
	.seq_adjust	= nf_nat_sip_seq_adjust,
	.expect		= nf_nat_sip_expect,
	.sdp_addr	= nf_nat_sdp_addr,
	.sdp_port	= nf_nat_sdp_port,
	.sdp_session	= nf_nat_sdp_session,
	.sdp_media	= nf_nat_sdp_media,
};

static int __init nf_nat_sip_init(void)
{
	BUG_ON(nf_nat_sip_hooks != NULL);
	nf_nat_helper_register(&nat_helper_sip);
	RCU_INIT_POINTER(nf_nat_sip_hooks, &sip_hooks);
	nf_ct_helper_expectfn_register(&sip_nat);
	return 0;
}

所以要进入这个函数，首先添加一条DNAT规则，用户态直接通过iptables操作：

1
# iptables -t nat -A PREROUTING -p udp -s 10.103.240.222 --dport 5060 -j DNAT --to-destination 1.2.3.4

iptable指定-t nat选项查看添加完成后的NAT规则：

1
2
3
4
5
6
7
8
9
10
11
12
13
# iptables -nvL -t nat
Chain PREROUTING (policy ACCEPT 18234 packets, 3134K bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DNAT       udp  --  *      *       10.103.240.222       0.0.0.0/0            udp dpt:5060 to:1.2.3.4

Chain INPUT (policy ACCEPT 16998 packets, 2947K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 269 packets, 109K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 277 packets, 110K bytes)
 pkts bytes target     prot opt in     out     source               destination

加载nf_nat_sip模块，打开内核报文转发以及netfilter对应的helper功能：

1
2
3
# modprobe nf_nat_sip
# echo 1 > /proc/sys/net/ipv4/ip_forward
# echo 1 > /proc/sys/net/netfilter/nf_conntrack_helper

使用scapy构造异常报文发送：

1
2
3
4
>>> payload = '425945207369703a736572766963654030303937302e312e312e313a35303630205349502f322e300d0a5669613a205349502f322e302f5544502031302e3130332e3234302e3233323a353036303b6272616e63683d7a39684734624b2d313032373537382d313835392d390d0a46726f6d3a2073697070203c7369703a736970704031302e3130332e3234302e3233323a353036303e3b7461673d313835390d0a546f3a20737574203c7369703a736572766963654030303937302e312e312e313a353036303e3b7461673d3132393736534950705461673031313835390d0a43616c6c2d49443a20313835392d313032373537384031302e3130332e3234302e3233320d0a435365713a2032204259450d0a436f6e746163743a207369703a736970704031302e3130332e3234302e3233323a353036300d0a4d61782d466f7277617264733a2037300d0a5375626a6563743a20506572666f726d616e636520546573740d0a436f6e74656e742d4c656e6774683a20300d0a0d0a'.decode('hex')
>>> send(IP(dst="10.103.241.233")/UDP(sport=5060,dport=5060)/payload)
.
Sent 1 packets.

再通过kprobe输出in4_pton的参数：

1
2
3
4
5
6
7
8
9
10
11
12
13
# ./kprobe 'p:in4_pton src=+0(%di):string'
				<idle>-0     [002] ..s. 858059.345971: in4_pton: (in4_pton+0x0/0x170) src="00970.1.1.1:5060 SIP/2.0
Via: SIP/2.0/UDP 10.103.240.232:5060;branch=z9hG4bK-1027578-1859-9
From: sipp <sip:sipp@10.103.240.232:5060>;tag=1859
To: sut <sip:service@00970.1.1.1:5060>;tag=12976SIPpTag011859
Call-ID: 1859-1027578@10.103.240.232
CSeq: 2 BYE
Contact: sip:sipp@10.103.240.232:5060
Max-Forwards: 70
Subject: Performance Test
Content-Length: 0

"

从kprobe输出中可以看到，内核in4_pton的参数src也是若干0开头的字符串，按照对代码的分析会溢出，但为啥内核还是好好的？是不是内核溢出不会导致coredump？

据此，编写一个溢出的内核模块：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
#include <linux/module.h>
#include <linux/kernel.h>

static int init(void)
{
        int i, w, c;

        printk("mo loaded.\n");

        w = 0;
        c = 0x30000;

        for (i = 0; i < 30; i++) {
                w = w * 10 + c;
                printk("w = 0x%08x\n", w);
        }

        return 0;
}

static void fini(void)
{
        printk("mo unloaded.\n");
}

module_init(init);
module_exit(fini);
MODULE_LICENSE("GPL");

加载模块后观察日志输出：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
2022-12-16 15:50:18.346619 warning [kernel:] [195343.614466] mo loaded.
2022-12-16 15:50:18.346629 warning [kernel:] [195343.614467] w = 0x00030000
2022-12-16 15:50:18.346630 warning [kernel:] [195343.614468] w = 0x00210000
2022-12-16 15:50:18.346631 warning [kernel:] [195343.614468] w = 0x014d0000
2022-12-16 15:50:18.346631 warning [kernel:] [195343.614468] w = 0x0d050000
2022-12-16 15:50:18.346631 warning [kernel:] [195343.614469] w = 0x82350000
2022-12-16 15:50:18.346632 warning [kernel:] [195343.614469] w = 0x16150000
2022-12-16 15:50:18.346632 warning [kernel:] [195343.614469] w = 0xdcd50000
2022-12-16 15:50:18.346639 warning [kernel:] [195343.614470] w = 0xa0550000
2022-12-16 15:50:18.346639 warning [kernel:] [195343.614470] w = 0x43550000
2022-12-16 15:50:18.346639 warning [kernel:] [195343.614470] w = 0xa1550000
2022-12-16 15:50:18.346640 warning [kernel:] [195343.614471] w = 0x4d550000
2022-12-16 15:50:18.346640 warning [kernel:] [195343.614471] w = 0x05550000
2022-12-16 15:50:18.346640 warning [kernel:] [195343.614471] w = 0x35550000
2022-12-16 15:50:18.346641 warning [kernel:] [195343.614472] w = 0x15550000
2022-12-16 15:50:18.346641 warning [kernel:] [195343.614472] w = 0xd5550000
2022-12-16 15:50:18.346642 warning [kernel:] [195343.614472] w = 0x55550000
2022-12-16 15:50:18.346642 warning [kernel:] [195343.614473] w = 0x55550000
2022-12-16 15:50:18.346643 warning [kernel:] [195343.614473] w = 0x55550000
2022-12-16 15:50:18.346643 warning [kernel:] [195343.614473] w = 0x55550000
2022-12-16 15:50:18.346643 warning [kernel:] [195343.614473] w = 0x55550000
2022-12-16 15:50:18.346644 warning [kernel:] [195343.614474] w = 0x55550000
2022-12-16 15:50:18.346644 warning [kernel:] [195343.614474] w = 0x55550000
2022-12-16 15:50:18.346645 warning [kernel:] [195343.614474] w = 0x55550000
2022-12-16 15:50:18.346645 warning [kernel:] [195343.614475] w = 0x55550000
2022-12-16 15:50:18.346646 warning [kernel:] [195343.614475] w = 0x55550000
2022-12-16 15:50:18.346646 warning [kernel:] [195343.614475] w = 0x55550000
2022-12-16 15:50:18.346647 warning [kernel:] [195343.614475] w = 0x55550000
2022-12-16 15:50:18.346647 warning [kernel:] [195343.614476] w = 0x55550000
2022-12-16 15:50:18.346648 warning [kernel:] [195343.614476] w = 0x55550000
2022-12-16 15:50:18.346648 warning [kernel:] [195343.614476] w = 0x55550000

结果看上去确实溢出了，但没有造成coredump。

__mulvsi3

回过头看最初造成coredump的函数__mulvsi3，该函数是GCC内置函数，对应的汇编代码为：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
00000000004005f0 <__mulvsi3>:
  4005f0:       48 63 c7                movslq %edi,%rax
  4005f3:       48 63 f6                movslq %esi,%rsi
  4005f6:       48 0f af c6             imul   %rsi,%rax /* rax = rax * rsi */
  4005fa:       48 89 c2                mov    %rax,%rdx
  4005fd:       89 c1                   mov    %eax,%ecx
  4005ff:       48 c1 fa 20             sar    $0x20,%rdx
  400603:       c1 f9 1f                sar    $0x1f,%ecx
  400606:       39 d1                   cmp    %edx,%ecx /* rdx >> 0x20 == ecx >> 0x1f */
  400608:       75 02                   jne    40060c <__mulvsi3+0x1c>
  40060a:       f3 c3                   repz retq 
  40060c:       50                      push   %rax
  40060d:       e8 1e fe ff ff          callq  400430 <abort@plt>
  400612:       66 2e 0f 1f 84 00 00    nopw   %cs:0x0(%rax,%rax,1)
  400619:       00 00 00 
  40061c:       0f 1f 40 00             nopl   0x0(%rax)

可以看到在计算完乘积后，通过最高位的进位和次高位的进位情况进行溢出判断（双高位判断法，原理没搞懂），如果发生溢出则跳转，并且最终执行abort。

为什么内核没有发生coredump，大概因为内核的乘法对应的实现没有做溢出检测。应用层也是同理，如果对应的乘法实现没有溢出检测的话也不会出现coredump。

通常来说，整数溢出的问题主要可能影响一些条件判断，进而造成死循环等问题，这种导致coredump的比较少，但总的来说还是应该尽量避免。

  • Copyright： Copyright is owned by the author. For commercial reprints, please contact the author for authorization. For non-commercial reprints, please indicate the source.
  • Copyrights © 2021-2023 Martzki
  • Visitors: | Views:

请我喝杯咖啡吧~

支付宝
微信