Aros/Developer/Docs/Libraries/BSDsocket

Introduction
Amiga had a few attempts at Internet access (TCP) and address (IP) stacks. AmiTCP (BSD 4.3) to Miami to Roadshow (OS4 and possibly AmigaOS 3.x (TM)).

"Miami" added its own APIs without extending the fundamental AmiTCP API functions, for example. As for Roadshow, piggy-back its own extended API on the AmiTCP foundations. Coordinating development on making all AmiTCP-like TCP/IP stacks share common API functionality is possibly doomed to fail.

The API we have today is used by practically all available Amiga networking software. It's reasonably well-documented, as it conforms to the basic BSD socket API functionality, which has not changed radically in the last 20 years.

What has changed in the last 20 years is the DNS lookup API. The version cast into bsdsocket.library is simple and close to the original, as first deployed in the late 1980s/early 1990s. This API still works, it's just not as convenient as one might want it to be.

None of this has to be replaced because it lacks capabilities which are vital to software operations.

Applications are built using AmiTCP's API (Application Programmers Interface) and don't care what goes on beneath them. Similarly, AmiTCP doesn't care what kind of hardware it's using, it just talks to the SANA-II interface of the device you specify. The SANA-II devices talk almost directly to the devices they abstract.


 * App
 * 1) AmiTCP and others like UDP IP etc.
 * 2) SANA-II devices
 * 3) Hardware

The layered approach for networking.

SANA is a Network Device Driver Specification written by Commodore which provides an Amiga software interface between networking hardware and network protocol stacks such as AmiTCP. What it means for you is that AmiTCP can run IP over any type of network for which there exists a SANA-II driver.

Usage
Using bsdsocket.library is not hard, you need to add the AmiTCP-SDKs in your include-path and include ,  and . It's like using any other shared library - you can utilize its functions after opening it with exec.library/OpenLibrary. After that, it is more or less identical to programming the BSD sockets API, as far as the networking is concerned.

create socket set sock addr set sock port open socket send socket ( request_string ) receive result

then write the input buffer to standard output or to a file

A few of these functions have macros for easier use (select, inet_ntoa, etc.), but you must use CloseSocket and IoctlSocket in place of close and ioctl. Bsdsocket.library has its own ernno, which can be queried with Errno or you can make it use the errno variable using SetErrnoPtr or SocketBaseTagList

Before calling the bsd functions, make sure the bsdsocket.library is opened, if not then its very normal that is where it crashes...

net_udp.c contains...
 * 1) include 
 * 2) include 
 * 3) include 
 * 4) include 
 * 5) include 
 * 6) include 
 * 7) include 


 * 1) include 
 * 2) include 
 * 3) include 


 * 1) include 
 * 2) include 
 * 3) include 
 * 4) include <netdb.h>


 * 1) ifdef __AROS__
 * 2) include proto/bsdsocket.h
 * 3) endif


 * 1) ifdef NeXT
 * 2) include libc.h
 * 3) endif

and uncomment them //extern int gethostname (char *, int); //extern int close (int);

For example, you create a socket with socket (from bsdsocket.library) and then you pass it to write from arosc, but this integer value means something different to arosc. Same with other way around, using fd created by arosc with bsdsocket.library.

Every bsdsocket.library call takes SocketBase as an argument, and since you declared a global NULL SocketBase socket functions will fail. To solve the problem, you can include the header below instead of proto/bsdsocket.h, and use SocketBase as normal, which will become the taks's user defined field. Please note that you have to do call init_arossock in the right thread, preferably in a wrapper around exec_list, and don't declare a global SocketBase variable.


 * 1) ifndef TASKSOCKBASE_H
 * 2) define TASKSOCKBASE_H

/* * Per-task socketbase using tc_UserData */


 * 1) include <proto/exec.h>


 * 1) define SocketBase FindTask(NULL)->tc_UserData
 * 2) define __BSDSOCKET_NOLIBBASE__
 * 3) include <proto/bsdsocket.h>


 * 1) endif

As far as i know WaitSelect works just like select and would be nice to paste bit of the code here where it fails... WaitSelect can only be used on sockets, not on DOS filehandles. On unix they are handled uniformly, but on Amiga you have to use different code path for file handles and sockets.

sys/select.h ---
 * 1) ifndef select(nfds,rfds,wfds,efds,timeout)
 * 2) define select(nfds,rfds,wfds,efds,timeout) WaitSelect(nfds,rfds,wfds,efds,timeout,NULL)
 * 3) endif

and the test the select

struct timeval timeout; int socket_d; struct fd_set client_d;

//set to 3 mins timeout.tv_sec = 3 * 60; timeout.tv_usec = 0;

do {

rc = select(socket_d + 1, &client_d, NULL, NULL, &timeout);

/* Check to see if the select call failed. */ if (rc < 0) { perror(" select failed"); break; }

/* Check to see if the 3 minute time out expired. */ if (rc == 0) { printf(" select timed out. End program.\n"); break; } }

One of the arguments of LockMutex is still NULL, it could be ThreadBase too. One thing I noticed is that you don't use pthread_join/WaitThread in your code. Instead you do busy loops until the given thread sets its state. Because one of the functions which overflows the stack is wait_thread_running.

LockMutex fails in exe_elist, because cclist_mutex is not initialized for AROS in main.c, hence the NULL pointer.

Initializing cclist_mutex solves the problem. Here are the two thread-waiting functions without the busy loop:

void wait_thread_running (S4 threadnum) {   LockMutex (pthreads_mutex); if (pthreads[threadnum].thread) WaitThread(pthreads[threadnum].thread, NULL); UnlockMutex (pthreads_mutex); }

void join_threads (S4 threadnum) {   /* wait until all threads are stopped */ S4 i;   LockMutex (pthreads_mutex); /* thread zero is the main thread, don't wait for it!!! */   for (i = 1; i < MAXPTHREADS; i++) {       /* don't check the calling thread!!! */       if (i != threadnum && pthreads[i].thread) {           WaitThread(pthreads[i].thread, NULL); }   }    UnlockMutex (pthreads_mutex); }

Replacing WaitThread with pthread_join would work on other platforms. Checking (pthreads[i].state == PTHREAD_RUNNING) isn't necessary anymore, since these thread waiting functions instantly return on terminated threads. It's waiting till the thread is started (was created). It's the thread_sync function in the examples. This is for synchronizing threads. The next step would be sending data to the thread: with threadpush.

Also make sure that to open bsdsocket.library in the same thread where you do the socket operations, because you can't share socket descriptors between threads. Technically it's possible to pass sds between threads, but it requires some extra code, and generally not worth the hassle if you can move all the socket manipulation into one thread.

Do you  know that bsdsocket API has support for fd hooks. They allow to sync up fd numbers between bsdsocket.library and libc. If you look at original  netlib code, you'll see it. That implementation is based on some support from SAS/C libc.

struct Library * SocketBase = NULL; int h_errno = 0; extern int errno;
 * 1) include <proto/exec.h>

and then in the initialization code:

if(!(SocketBase = OpenLibrary("bsdsocket.library", 4 ))) { SDLNet_SetError("No TCP/IP Stack running!\n"); return(-1); }

if( SocketBaseTags(SBTM_SETVAL(SBTC_ERRNOPTR(sizeof(errno))), (IPTR)&errno, SBTM_SETVAL(SBTC_HERRNOLONGPTR), (IPTR)&h_errno, TAG_DONE )) { SDLNet_SetError("Error initializing bsdsocket\n"); return(-1); }


 * Under AROS you need -larossupport instead of -ldebug.
 * If using SDL and its' functions, you will need -lsdl
 * make sure ioctl or ioctlsocket is defined to IoctlSocket, and close to CloseSocket.

AmiTCP contains the bare necessities to perform DNS lookups through an early version of the libresolv code, which is baked into it. Problem here is that you cannot retrofit this API. You might be able to emulate it. You might be able to use conditional compilation to have the client code call the old clunky API instead of the shinier version we don't have. You might be able to simply drop in a more recent version of the libresolv code than what exists as part of the AmiTCP API.

A lot has changed within the socket API in the past decades, and the same is true for the libresolv code. New functions and data structures were tacked on which make accessing the fundamental API services less clunky and more convenient. These niceties passed us by, because the AmiTCP API has remained virtually unchanged since its inception.

Future
The first is about breaking some api level functionality to make way for supporting IPv6. Would it be a great deal to fix the existing functions that assume IPv4 addresses, so they take generic structures instead? miami.library provides the missing API. Miami was going the same way back then. IPv6 looks like a failed experiment. No one seriously deploys it. In fact, not much needs to be replaced. It affects mainly resolver, introducing inet_PtoN and inet_NtoP. The rest is perfectly handled by existing socket API. Just add AF_INET6.

How big a task would it be to replace the existing bsd-based internals of AROSTCP with more up to date code? Take recent NetBSD and you're done. FreeBSD diverted a lot. This provides working sysctl and BPF/PCAP. AROSTCP already has some upgraded code. Its codebase is NetBSD v2.0.5. BTW, userland (like resolver) from FreeBSD perfectly fits in. One more idea: separate protocols into loadable modules. This would give you AF_UNIX, AF_BT, etc. There is a single-linked list where all protocols are registered. Nodes are searched using AF_xxx as a key. Then there are several pointers to functions. That's all. This is separable. In the end you would have protocol modules with these functions exported, and something like bsdcore.library, implementing basic low-level stuff like mbufs and bsd_malloc. There is remaining (IIRC) from original BSD4.3, where AF_xxx are indexes into array. I kept this. So, for now both lookup mechanisms are coexisting. Discard the old one, use NetBSD code as a reference. In fact BSD is very modular at source code level, the same as Linux. You can switch on and off any part without affecting others. So, porting the code isn't difficult.

Would like to propose a 'unified file descriptor' model for arosc.library, to simplify porting of POSIX applications. The goal is to be able to use the same 'int fd' type to represent both the DOS BPTR filehandles and bsdsocket.library socket back-end. This will eliminate the need to rewrite code to use send instead of write to sockets, and call CloseSocket instead of close for sockets.

Implementation would be as a table (indexed by fd #) that would have the following struct:

struct arosc_fd { APTR file; struct arosc_fd_operation {

/* Required for all FDs */     int (*close)(APTR file); LONG (*sigmask)(APTR file); /* Gets the IO signal mask for poll(2)ing */ ssize_t (*write)(APTR file, const void *buff, size_t len); ssize_t (*read)(APTR file, void *buff, size_t len); ..etc etc...

/* The following are optional operations */     off_t (*lseek)(APTR file, off_t offset, int whence); ssize_t (*sendto)(APTR file, const void *buff, size_t len, int flags, const struct sockaddr *dest_addr, socklen_t addrlen); ssize_t (*recvfrom)(APTR file, void *buff, size_t len, int flags, struct sockaddr *src_addr, socklen_t *addrlen); ssize_t (*sendmsg)(APTR file, const struct msghdr *msg, int flags); ssize_t (*recvmsg)(APTR file, struct msghdr *msg, int flags); ..etc etc.. } *op; } *fd_table;

The C library routines 'open' and 'socket' would be modified to:


 * Ensure that either dos.library or bsdsocket.library is opened.
 * Allocate a new fd in the FD table
 * Set fd_table[fd].ops to a 'struct arosc_fd_operation' pointer, that defines the IO operations for either dos.library for bsdsocket.library
 * Set fd_table[fd].file to a pointer to the private data for those operation (i.e. BPTR or socket fd #)
 * Return the new FD number

The read/write/etc. would be modified to use the fd_table[fd].op routines to perform IO

The poll routine will wait for activity on the OR of all the signal masks of the provided FDs.

The close routine also deallocates the arosc_fd entry

It's a layer on top of it. bsdsocket.library would be opened dynamically from arosc.library's socket routine if needed.

Since arosc generated FD's will be different from bsdsocket.library ones how will programmers know what to pass to functions? Programmers would not use the bsdsocket headers nor protos directly (similar arosc.library headers would be generated that wrap them). The point is that programmers would no longer need to treat bsdsocket fd as a special case.

the whole fd_table[] mapping concept. The 'file' element of the fd_table[n] would be a BPTR for an opened fd, and a bsdsocket.library socket descriptor number for one created by arosc's socket call. The user would *not* be calling bsdsocket.library directly under the proposed scheme. Everything in that library would be wrapped by arosc.library.

make sure that those codes will continue using bsdsocket functions and won't switch half arosc / half bsdsocket (for example arosc->send and bsdsocket->CloseLibrary). The 'best' solution would be to have a compilation flag (i.e.

'#define BSDSOCKET_API') that ifdefs out the arosc send/recv/socket/etc prototypes and makes send/recv/CloseSocket and friends from the bsdsocket protos visible.

If BSDSOCKET_API is not defined, but bsdsocket.library is intended (i.e. calls to CloseSocket, etc.), then the user will get compilation errors about missing function prototypes. If BSDSOCKET_API is defined, but POSIX is intended, then they will get a link error about 'undefined symbol SocketLibrary'

Maybe the bsdsocket includes should also have some detection of arosc includes and produce the warning about the BSDSOCKET_API define being accessible to developers if both headers are included in the same compilation unit - I'm just wondering how we are going to get the word out to third-party developers that they don't have to change their codes and can simply use the define - or maybe it should be the other way around? bsdsocket by default and POSIX_SOCKET_API define to enable the support you mention?

As for the stubs/monkey wrenching, this won't fly due to how the bsdsocket.library architecture works out. Each time you open the library, it creates state information specific to the opener. This state information ties into the original BSD kernel's process data structure. It's fundamentally unsound to even try patching that.