I have used RSAX_BIW_GET_DATA_SIMPLE FM for my generic datasource and the code logic is standard i.e i replaced existing logic with my logic for data fetching and bringing it in E_T_DATA.
Just have a look at the code and let me know if i am missing anything.
TABLES: eban.
DATA: L_S_SELECT TYPE SRSC_S_SELECT.
STATICS: S_S_IF TYPE SRSC_S_IF_SIMPLE,
S_COUNTER_DATAPAKID LIKE SY-TABIX,
S_CURSOR TYPE CURSOR.
* Select ranges
RANGES: L_R_banfn FOR eban-banfn,
L_R_werks FOR eban-werks,
L_R_badat FOR eban-badat,
L_R_erdat FOR eban-erdat.
DATA: LWA_DATA LIKE LINE OF E_T_DATA.
* Initialization mode (first call by SAPI) or data transfer mode
* (following calls) ?
IF I_INITFLAG = SBIWA_C_FLAG_ON.
APPEND LINES OF I_T_SELECT TO S_S_IF-T_SELECT.
* Fill parameter buffer for data extraction calls
S_S_IF-REQUNR = I_REQUNR.
S_S_IF-DSOURCE = I_DSOURCE.
S_S_IF-MAXSIZE = I_MAXSIZE.
* Fill field list table for an optimized select statement
* (in case that there is no 1:1 relation between InfoSource fields
* and database table fields this may be far from beeing trivial)
APPEND LINES OF I_T_FIELDS TO S_S_IF-T_FIELDS.
ELSE. "Initialization mode or data extraction ?
************************************************************************
* Data transfer: First Call OPEN CURSOR + FETCH
* Following Calls FETCH only
************************************************************************
* First data package -> OPEN CURSOR
IF S_COUNTER_DATAPAKID = 0.
* Fill range tables BW will only pass down simple selection criteria
* of the type SIGN = 'I' and OPTION = 'EQ' or OPTION = 'BT'.
LOOP AT S_S_IF-T_SELECT INTO L_S_SELECT WHERE FIELDNM = 'BANFN'.
MOVE-CORRESPONDING L_S_SELECT TO L_R_BANFN.
APPEND L_R_BANFN.
ENDLOOP.
LOOP AT S_S_IF-T_SELECT INTO L_S_SELECT WHERE FIELDNM = 'WERKS'.
MOVE-CORRESPONDING L_S_SELECT TO L_R_WERKS.
APPEND L_R_WERKS.
ENDLOOP.
LOOP AT S_S_IF-T_SELECT INTO L_S_SELECT WHERE FIELDNM = 'BADAT'.
MOVE-CORRESPONDING L_S_SELECT TO L_R_BADAT.
APPEND L_R_BADAT.
ENDLOOP.
LOOP AT S_S_IF-T_SELECT INTO L_S_SELECT WHERE FIELDNM = 'ERDAT'.
MOVE-CORRESPONDING L_S_SELECT TO L_R_ERDAT.
L_R_ERDAT-SIGN = 'I'.
L_R_ERDAT-OPTION = 'GE'.
CLEAR L_R_ERDAT-HIGH .
APPEND L_R_ERDAT.
ENDLOOP.
OPEN CURSOR WITH HOLD S_CURSOR FOR
SELECT banfn bnfpo ekgrp txz01 matnr matkl werks lgort menge meins badat lfdat erdat bsmng ebakz frggr
FROM eban
WHERE BANFN IN L_R_BANFN
AND WERKS IN L_R_WERKS
AND BADAT IN L_R_BADAT
AND ERDAT IN L_R_ERDAT
AND pstyp = '0'
AND loekz = ' '.
ENDIF. "First data package ?
* Fetch records into interface table.
* named E_T_'Name of extract structure'.
FETCH NEXT CURSOR S_CURSOR
APPENDING CORRESPONDING FIELDS
OF TABLE gt_eban
PACKAGE SIZE S_S_IF-MAXSIZE.
SORT GT_EBAN BY BADAT.
IF SY-SUBRC <> 0.
CLOSE CURSOR S_CURSOR.
RAISE NO_MORE_DATA.
ELSE.
IF gt_eban[] IS NOT INITIAL.
Here I have written further selection logic and brought the final data in E_T_DATA.
MOVE-CORRESPONDING <gs_list> TO LWA_DATA.
APPEND LWA_DATA TO E_T_DATA.
ENDLOOP.
REFRESH : gt_list[].
ENDIF.
S_COUNTER_DATAPAKID = S_COUNTER_DATAPAKID + 1.
ENDIF. "Initialization mode or data extraction ?
My issue is, In RSA3 when I check data then different records are coming in each data package but when I am doing full load in BI side. It is bringing duplicate records in different datapackage as below
DP Records
| 000001 | 18627 |
| 000002 | 37341 |
| 000003 | 56282 |
| 000004 | 75208 |
| 000005 | 93910 |
| 000006 | 112431 |
In DP2 I am getting all records of DP1 and in DP3 I am getting all records of DP2 and so on.
In total I have approx 500000 records.